Software Development Engineer in Test - AI - Palo Alto, United States
$140,000 - $220,000 USD yearly
Originally posted 31 August 2026 by the employer.
About the role
This role involves ensuring the reliability and performance of core AI infrastructure by building sophisticated automation frameworks and stress-testing suites.
What you'll do
- Design, develop, and maintain a robust automated testing framework supporting distributed AI training and inference workloads.
- Develop complex test plans focusing on end-to-end system integration, stress testing, and hardware-software boundary conditions.
- Partner with System Engineers to debug deep-seated issues in distributed clusters, using telemetry and profiling tools to identify bottlenecks.
What you'll need
- Strong proficiency in Python for automation and orchestration.
- Proven experience building or extending test automation frameworks for complex back-end systems.
- Proven ability to troubleshoot automated test failures within complex large-scale distributed systems and identify root causes.
- Experience with containerization (Docker/Kubernetes) and modern CI/CD tools (GitHub Actions, GitLab CI, or Jenkins).
Nice to have
- Experience with Kubernetes or Terraform or Ansible for managing test-bed environments.
- Experience testing high-performance networking protocols or distributed file systems.
- Experience testing software that interacts directly with drivers or firmware.
Skills: automated testing framework, distributed AI training, GPU kernels, AI frameworks, distributed systems
This role has been open 47 days — below the 51-day median for Infrastructure/Platform Software roles.
Infrastructure/Platform Software · Infrastructure Platform
|
Open roles in category
962
|
Median days open
51 d
|
Median salary
$236k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | Delos Data | All employers we track in this specialty (962 roles · 86 employers) |
|---|---|---|
| Open roles in this specialty | 6 | 962 |
| Open roles in the wider Software, Firmware & Systems family | 7 | 6118 · 145 employers |
| Median days open | 47 d | 51 d (−4 d vs this employer) |
| Median salary (USD postings) | — | $236k |
Skills observed across this category: automated testing framework, distributed AI training, GPU kernels, AI frameworks, distributed systems
Who's hiring in this category
- NVIDIA · 274 open roles · median 47 d
- Qualcomm · 62 open roles · median 53 d
- AMD · 54 open roles · median 19 d
- Graphcore · 44 open roles · median 93 d
- Intel Corporation · 43 open roles · median 12 d
- Cerebras · 36 open roles · median 83 d
How we counted: 962 open Infrastructure/Platform Software (Infrastructure Platform) roles from 86 employers tracked in the SemiconductorJobs index, counted 18 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.