Software Engineer, GPU Kernels - Palo Alto, CA, United States
$200,000 - $420,000 USD yearly
Originally posted 12 September 2026 by the employer.
Work on GPU kernels for River's training and inference infrastructure.
About the role
This role involves building compute primitives for River's training and inference infrastructure. You will work on performance-critical operations such as attention, matrix multiplication, and mixture-of-experts execution.
What you'll do
- Build fast GPU kernels for attention, matrix multiplication, expert routing, and related operations.
- Optimize memory access, tiling, and synchronization for efficient GPU hardware utilization.
- Develop FP8, FP4, and mixed-precision kernels while preserving numerical correctness.
- Profile real workloads and integrate improvements into training and inference runtimes.
- Build reproducible benchmarks that verify correctness, gradients, and performance.
What you'll need
- Bachelor’s degree in Computer Science, Computer Engineering, or equivalent practical experience.
- Experience optimizing GPU kernels with CUDA, Triton, CUTLASS, CuTe, or comparable tools.
- Strong understanding of GPU architecture, memory hierarchies, and parallel execution.
- Proficiency in C++ and Python.
- Strong foundations in linear algebra, floating-point arithmetic, and numerical computing.
- Strong debugging and profiling skills.
Nice to have
- Experience optimizing for NVIDIA Blackwell or Hopper GPUs.
- Work on attention, mixture-of-experts kernels, grouped GEMMs, or low-rank adapters.
- Experience implementing backward passes and validating gradients.
- Familiarity with FP8, FP4, and quantized weight layouts.
- Experience integrating custom operators into PyTorch, SGLang, vLLM, or similar frameworks.
- Open-source contributions or a track record of shipping kernel optimizations.
Skills: GPU kernels, CUDA, Triton, CUTLASS, memory access
This role has been open 33 days — well below the 56-day median for Infrastructure/Platform Software roles.
Infrastructure/Platform Software · GPU Software Stack
|
Open roles in category
231
|
Median days open
56 d
|
Median salary
$236k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | River | All employers we track in this specialty (231 roles · 20 employers) |
|---|---|---|
| Open roles in this specialty | 2 | 231 |
| Open roles in the wider Software, Firmware & Systems family | 7 | 6145 · 145 employers |
| Median days open | 77 d | 56 d (+21 d vs this employer) |
| Median salary (USD postings) | — | $236k |
Skills observed across this category: GPU kernels, CUDA, Triton, CUTLASS, memory access
Who's hiring in this category
- NVIDIA · 117 open roles · median 55 d
- AMD · 53 open roles · median 65 d
- Qualcomm · 19 open roles · median 106 d
- Intel Corporation · 17 open roles · median 6 d
- Arm Holdings · 5 open roles · median 112 d
- d-Matrix · 2 open roles · median 120 d
How we counted: 231 open Infrastructure/Platform Software (GPU Software Stack) roles from 20 employers tracked in the SemiconductorJobs index, counted 15 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.