Inference Intern - San Jose, United States
Originally posted 8 December 2025 by the employer.
About the role
This role focuses on developing and optimizing compute architectures for inference workloads as an intern at Etched.
What you'll do
- Assist in building, enhancing, and scaling the runtime, including multi-node inference, intra-node execution, state management, and robust error handling. - Contribute to optimizing routing and communication layers using internal collectives. - Utilize performance profiling and debugging tools to identify bottlenecks and correctness issues. - Co-design both HW instructions and model architecture operations to maximize model performance. - Implement high-performance software components for the Model Toolkit.
What you'll need
- Progress towards a Bachelor’s, Master’s, or PhD degree in computer science, computer engineering, applied mathematics, or a related field. - Proficiency in Python and C++. - Understanding of performance-sensitive or complex distributed software systems, such as Linux internals, accelerator architectures (e.g., GPUs, TPUs), compilers, or high-speed interconnects (e.g., NVLink, InfiniBand). - Experience porting applications to non-standard accelerator hardware or hardware platforms. - Deep knowledge of transformer model architectures and/or inference serving stacks (e.g., vLLM, SGLang).
Skills: porting state-of-the-art models, programming abstractions, runtime, multi-node inference, HW instructions, model architecture operations
This role has been open 219 days — well above the 62-day median for Infrastructure/Platform Software roles.
Infrastructure/Platform Software · AI ML Hardware
|
Open roles in category
105
|
Median days open
62 d
|
Median salary
$216k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | Etched | All employers we track in this specialty (105 roles · 28 employers) |
|---|---|---|
| Open roles in this specialty | 2 | 105 |
| Open roles in the wider Software, Firmware & Systems family | 42 | 6291 · 145 employers |
| Median days open | 219 d | 62 d (+157 d vs this employer) |
| Median salary (USD postings) | — | $216k |
Skills observed across this category: porting state-of-the-art models, programming abstractions, runtime, multi-node inference, HW instructions, model architecture operations
Who's hiring in this category
- Qualcomm · 25 open roles · median 52 d
- NVIDIA · 18 open roles · median 58 d
- Graphcore · 8 open roles · median 118 d
- AMD · 7 open roles · median 51 d
- Intel Corporation · 6 open roles · median 25 d
- Micron Technology · 6 open roles · median 45 d
How we counted: 105 open Infrastructure/Platform Software (AI ML Hardware) roles from 28 employers tracked in the SemiconductorJobs index, counted 22 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.