Principal System Software Engineer, AI Inference Execution - Santa Clara, United States
Originally posted 15 May 2026 by the employer.
About the role
This role helps productize the software stack for d-Matrix's AI compute engine. You will be responsible for the development, enhancement, and maintenance of AI deployment software across the full-stack toolchain.
What you'll do
- Productize the software stack for the AI compute engine. - Lead the development, enhancement, and maintenance of AI deployment software. - Optimize and trade off aspects of hardware-software co-design across the full-stack toolchain. - Build and scale software deliverables within tight development windows. - Build out deployment infrastructure in collaboration with ML, compiler, and hardware teams.
What you'll need
- BS in Computer Science, Engineering, Math, Physics, or related degree with 12+ years of industry software development experience, or MS with 6+ years. - Strong grasp of system software, data structures, computer architecture, and machine learning fundamentals. - Proficiency in C, C++, and Python development in a Linux environment using standard development tools. - Experience with distributed, high-performance software design and implementation. - Demonstrated ownership and technical leadership in system software development.
Nice to have
- MS or PhD in Computer Science, Electrical Engineering, or related fields. - Experience with inference servers or model serving frameworks such as TensorRT-LLM, vLLM, or SGLang. - Experience with deep learning frameworks such as PyTorch and TensorFlow. - Experience with deep learning runtimes such as ONNX Runtime and TensorRT. - Experience with distributed systems collectives such as NCCL and OpenMPI. - Experience with software testing fundamentals. - Experience deploying ML workloads such as LLMs, VLMs, and NLP on distributed systems. - Experience with Kubernetes, Ray, or other MLOps tools and techniques from definition to deployment. - Prior startup, small team, or incubation experience, or work experience at a cloud provider or AI compute/subsystem company.
Skills: SW stack, AI compute engine, deployment infrastructure, distributed systems, deep learning frameworks
This role has been open 129 days — well above the 56-day median for Infrastructure/Platform Software roles.
Infrastructure/Platform Software · GPU Software Stack
|
Open roles in category
242
|
Median days open
56 d
|
Median salary
$220k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | d-Matrix | All employers we track in this specialty (242 roles · 20 employers) |
|---|---|---|
| Open roles in this specialty | 1 | 242 |
| Open roles in the wider Software, Firmware & Systems family | 23 | 6298 · 144 employers |
| Median days open | 129 d | 56 d (+73 d vs this employer) |
| Median salary (USD postings) | — | $220k |
Skills observed across this category: SW stack, AI compute engine, deployment infrastructure, distributed systems, deep learning frameworks
Who's hiring in this category
- NVIDIA · 120 open roles · median 56 d
- AMD · 52 open roles · median 54 d
- Intel Corporation · 25 open roles · median 13 d
- Qualcomm · 18 open roles · median 127 d
- Arm Holdings · 5 open roles · median 120 d
- Graphcore · 4 open roles · median 2 d
How we counted: 242 open Infrastructure/Platform Software (GPU Software Stack) roles from 20 employers tracked in the SemiconductorJobs index, counted 23 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.
Similar Jobs
Infrastructure Systems Engineer - Arizona, Phoenix, United States
Intel Corporation · Remote friendly (US, Arizona, Phoenix, United States)
Infrastructure Storage Engineer - Arizona, Phoenix, United States
Intel Corporation · Remote friendly (US, Arizona, Phoenix, United States)