LLM Serving Engineer (Cloud AI Engineering), Senior / Staff Engineer - San Diego, California, United States of America
$158,400 - $237,600 USD yearly
Originally posted 9 April 2026 by the employer.
About the role
The LLM Serving Engineer will join the Qualcomm Cloud AI team to develop hardware and software solutions for inference acceleration. The role focuses on building a scalable LLM inference platform across the full product lifecycle, from research and development to commercial deployment.
What you'll do
- Build a scalable LLM inference platform using disaggregated serving, KV-Cache management, advanced parallelism, speculative algorithms, model optimization, and specialized kernels. - Contribute to the development of LLM serving packages such as vLLM, SGLang, TGI, Triton-Inference Server, Dynamo, and LLM-d. - Collaborate with internal compiler, firmware, and platform teams to drive solutions for customer deployments. - Identify optimization opportunities by analyzing advanced algorithms including attention mechanisms and MoEs. - Implement efficient serving mechanisms utilizing autoscaling, load balancing, and routing. - Engage with open-source serving communities to evolve runtime frameworks.
What you'll need
- Bachelor's degree in Computer Science, Engineering, Information Systems, or related field with 4+ years of relevant engineering experience (or Master's with 3+ years, or PhD with 2+ years). - Hands-on experience with LLM serving or orchestration packages such as Triton-Inference Server, vLLM, SGLang, Ollama, llm-d, KServe, LMCache, or MoonCake. - Deep understanding of foundational LLMs, VLMs, SLMs, and transformer-based architectures. - Experience developing language models using PyTorch and strong Python development skills for large-scale projects. - Solid background in computer science fundamentals including parallel and distributed programming, data structures, and algorithms. - Understanding of computer architecture, ML accelerators, in-memory processing, and distributed systems. - Experience analyzing, profiling, and optimizing deep learning workloads.
Nice to have
- Open-source contributions to GenAI packages. - Experience architecting and developing large-scale distributed systems. - High-level kernel design experience using PyTorch, CUDA, or Triton. - Knowledge of torch.compile or torchDynamo. - PhD in Computer Science, Computer Engineering, or Machine Learning.
Skills: LLM serving, inference platform, Cloud AI, Deep Learning, model optimization, open-source serving communities
This role has been open 200 days — well above the 61-day median for Software Engineering Semiconductor roles.
Software Engineering Semiconductor · AI ML Hardware
|
Open roles in category
158
|
Median days open
61 d
|
Median salary
$191k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | Qualcomm | All employers we track in this specialty (158 roles · 43 employers) |
|---|---|---|
| Open roles in this specialty | 21 | 158 |
| Open roles in the wider Software, Firmware & Systems family | 735 | 6250 · 144 employers |
| Median days open | 200 d | 61 d (+139 d vs this employer) |
| Median salary (USD postings) | — | $191k |
Skills observed across this category: LLM serving, inference platform, Cloud AI, Deep Learning, model optimization, open-source serving communities
Who's hiring in this category
- NVIDIA · 26 open roles · median 211 d
- Micron Technology · 25 open roles · median 39 d
- Qualcomm (this employer) · 21 open roles · median 199 d
- Infineon Technologies · 12 open roles · median 44 d
- AMD · 8 open roles · median 61 d
- Applied Materials · 4 open roles · median 95 d
How we counted: 158 open Software Engineering Semiconductor (AI ML Hardware) roles from 43 employers tracked in the SemiconductorJobs index, counted 25 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.
Similar Jobs
Career Accelerator Program - Software Engineer - Dallas, TX, United States
Texas Instruments · Dallas, TX, United States
Senior Unreal Engine Developer Technology Engineer - California, Folsom, United States
Intel Corporation · Remote friendly (US, California, Folsom, United States)