Fellow Software Development Engineer, GEMM Optimization - CA, San Jose, United States
Originally posted 2 September 2026 by the employer.
About the role
This role involves architecting and developing features in the tensile-lite assembly code generator for General Matrix Multiplication (GEMM) and GEMM + X, including attention. GEMM performance is critical for AI workload efficiency.
What you'll do
- Analyze AMD GPU hardware specifications in depth and work closely with hardware architects to align kernel design with current silicon capabilities.
- Influence requirements for future generations of hardware designs.
- Develop support for new ISA and HW/SW optimization features in the code-generator.
- Innovate and implement new algorithms for implementing GEMM and GEMM + X.
- Profile and root-cause GEMM performance bottlenecks across the stack.
- Partner with customers and internal stakeholders to understand real-world workload requirements, reproduce performance issues, and deliver targeted performance.
What you'll need
- Strong command of GPU computer architecture: compute units, register files, cache/LDS hierarchies, memory bandwidth, matrix cores/WMMA-style instructions, and instruction scheduling/latency hiding.
- Deep expertise in GEMM algorithms and their mapping onto GPU hardware (tiling, blocking, register/LDS allocation, wave scheduling, memory hierarchy utilization).
- Strong software engineering fundamentals with a track record of building production-quality, high-performance software.
- Demonstrated experience optimizing GPU compute kernels, ideally GEMM and attention.
- Proficiency in C++ and assembly-level GPU programming.
- Solid understanding of parallel computing, memory hierarchies, and hardware-software performance tradeoffs.
- Clear written and verbal communication skills; ability to work effectively with hardware architects, customers, and cross-functional engineering teams.
Nice to have
- Working knowledge of Python for tooling/automation.
- Direct experience with any GPU compiler backend.
- Experience with AMD GPU architectures (CDNA/RDNA) or comparable competitive architectures (NVIDIA Hopper/Blackwell, etc.).
- Contributions to industry standards, publications, or patents related to GPU compute or matrix-multiply optimization.
- BS/MS/PHD in CS/CE or related field with deep relevant experience.
Skills: GEMM, GPU architecture, assembly, code generation, AI workload, C++
This role has been open 43 days — well below the 56-day median for Infrastructure/Platform Software roles.
Infrastructure/Platform Software · GPU Software Stack
|
Open roles in category
231
|
Median days open
56 d
|
Median salary
$236k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | AMD | All employers we track in this specialty (231 roles · 20 employers) |
|---|---|---|
| Open roles in this specialty | 53 | 231 |
| Open roles in the wider Software, Firmware & Systems family | 404 | 6141 · 145 employers |
| Median days open | 64 d | 56 d (+8 d vs this employer) |
| Median salary (USD postings) | — | $236k |
Skills observed across this category: GEMM, GPU architecture, assembly, code generation, AI workload, C++
Who's hiring in this category
- NVIDIA · 117 open roles · median 55 d
- AMD (this employer) · 53 open roles · median 65 d
- Qualcomm · 19 open roles · median 106 d
- Intel Corporation · 17 open roles · median 6 d
- Arm Holdings · 5 open roles · median 112 d
- d-Matrix · 2 open roles · median 120 d
How we counted: 231 open Infrastructure/Platform Software (GPU Software Stack) roles from 20 employers tracked in the SemiconductorJobs index, counted 15 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.