Software Engineer- GPU/AI/ML - CA, Santa Clara, United States
$178,500 - $306,000 USD yearly
Originally posted 25 June 2026 by the employer.
About the role
This role improves how models train, align, and run on AMD GPUs, spanning from GPU kernels through distributed training and inference. You will work on software tuning for the ROCm ecosystem, foundation models, and agentic systems.
What you'll do
- Establish best practices and drive performance across the AI software stack, from low-level GPU kernels to large-scale distributed systems. - Improve training, post-training, and inference for LLMs and autonomous AI workloads. - Partner on hardware and software co-design across the lifecycle, from GPU architecture input to software for new accelerators. - Own technical problems, influence direction across teams, and mentor engineers.
What you'll need
- Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent experience. - Expert-level modern C++ and experience designing large, performance-critical systems. - Strong grasp of GPU architecture, memory hierarchy, and kernel optimization (HIP/CUDA). - Hands-on delivery on large-scale C++/HIP/CUDA codebases, such as ROCm (rocBLAS, hipDNN, Composable Kernel, AITemplate), the CUDA ecosystem (cuBLAS, cuDNN, CUTLASS, Thrust, CUB, NCCL), or ML framework cores such as PyTorch, TensorFlow, or JAX. - Experience diagnosing bottlenecks with profilers such as ROCm Profiler and Nsight in multi-GPU, distributed settings. - Deep understanding of transformers, attention, and the full model lifecycle, with hands-on work in alignment and post-training (such as SFT, RLHF, and GRPO). - Experience optimizing post-training and inference pipelines at scale, with knowledge of trends including MoE, quantization, and speculative decoding.
Nice to have
- Master's degree or PhD in Computer Science, Computer Engineering, Electrical Engineering, or a related field. - Deep familiarity with the AMD ROCm/HIP ecosystem. - Extensive HIP/CUDA experience optimizing deep learning and OSS LLM inference/training kernels and operators. - Working knowledge of RTL design and Verilog/SystemVerilog for hardware–software co-design. - Publications in AI/ML, GPU computing, or systems optimization.
Skills: GPU architecture, kernel optimization, ROCm, CUDA, LLMs, distributed training
This role has been open 119 days — well above the 56-day median for Infrastructure/Platform Software roles.
Infrastructure/Platform Software · GPU Software Stack
|
Open roles in category
238
|
Median days open
56 d
|
Median salary
$221k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | AMD | All employers we track in this specialty (238 roles · 20 employers) |
|---|---|---|
| Open roles in this specialty | 52 | 238 |
| Open roles in the wider Software, Firmware & Systems family | 400 | 6298 · 145 employers |
| Median days open | 54 d | 56 d (−2 d vs this employer) |
| Median salary (USD postings) | — | $221k |
Skills observed across this category: GPU architecture, kernel optimization, ROCm, CUDA, LLMs, distributed training
Who's hiring in this category
- NVIDIA · 117 open roles · median 56 d
- AMD (this employer) · 53 open roles · median 55 d
- Intel Corporation · 23 open roles · median 13 d
- Qualcomm · 18 open roles · median 126 d
- Arm Holdings · 5 open roles · median 119 d
- Graphcore · 4 open roles · median 1 d
How we counted: 238 open Infrastructure/Platform Software (GPU Software Stack) roles from 20 employers tracked in the SemiconductorJobs index, counted 22 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.