$150,000 - $350,000 USD yearly
Originally posted 4 November 2026 by the employer.
About the role
ChipAgents is seeking an ML Systems Engineer to optimize the performance and efficiency of large language model inference powering an agentic AI platform. The role focuses on low-level systems optimization, implementing performance improvements, building evaluation harnesses, and architecting multi-node clusters for training and inference.
What you'll do
- Design, deploy, and optimize LLM inference systems across multi-node clusters to maximize throughput and minimize latency for production workloads.
- Implement and benchmark concrete inference optimizations across the stack.
- Profile and analyze inference bottlenecks at the systems level, from GPU kernel execution to memory bandwidth constraints.
- Build evaluation harnesses and benchmarking frameworks measuring accuracy, throughput, latency, and resource utilization across parallelism strategies.
- Collaborate with research scientists to integrate new model architectures and optimizations into production inference infrastructure.
- Investigate and apply emerging techniques from research papers and open-source projects to improve inference performance.
What you'll need
- B.S., M.S., or PhD in Computer Science, Electrical Engineering, or a related field (or equivalent experience).
- Experience with large-scale ML systems, GPU computing, or high-performance inference optimization.
- Proficiency in Python and C++/CUDA, with hands-on experience in SGLang, vLLM, PyTorch, or similar inference frameworks.
- Understanding of GPU architecture, memory hierarchies, and parallel computing paradigms.
- Experience deploying and optimizing LLMs in production, including model serving, batching strategies, distributed inference, or quantization.
- Systems-level debugging and profiling skills across layers from CUDA kernels to application logic.
Nice to have
- Familiarity with distributed computing frameworks such as Ray, multi-node training, or multi-node inference.
Skills: CUDA, GPU kernel, LLM inference, PyTorch, vLLM, distributed inference
This role has been open 0 days — well below the 55-day median for Infrastructure/Platform Software roles.
Infrastructure/Platform Software · GPU Software Stack
|
Open roles in category
287
|
Median days open
55 d
|
Category median salary (all employers)
$229k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | ChipAgents | All employers we track in this specialty (287 roles · 21 employers) |
|---|---|---|
| Open roles in this specialty | 1 | 287 |
| Open roles in the wider Software, Firmware & Systems family | 2 | 6439 · 156 employers |
| Median days open | 0 d | 55 d (−55 d vs this employer) |
| Median salary (USD postings) | — | $229k |
Skills observed across this category: CUDA, GPU kernel, LLM inference, PyTorch, vLLM, distributed inference
Who's hiring in this category
- NVIDIA · 151 open roles · median 65 d
- AMD · 54 open roles · median 63 d
- Intel Corporation · 30 open roles · median 19 d
- Qualcomm · 17 open roles · median 194 d
- Arm Holdings · 9 open roles · median 7 d
- Graphcore · 4 open roles · median 14 d
How we counted: 287 open Infrastructure/Platform Software (GPU Software Stack) roles from 21 employers tracked in the SemiconductorJobs index, counted 4 Nov 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.