Sr Staff Engineer - San Jose, CA, United States
$220,800 USD yearly
Originally posted 16 September 2026 by the employer.
Design, build, and maintain an enterprise AI gateway.
About the role
As a Sr Staff Engineer, you will design and architect enterprise-wide Generative AI solutions, focusing on large language models and AI model consumption.
What you'll do
- Design and architect enterprise-wide Generative AI solutions, including reference architectures, integration patterns, and technical standards.
- Design, build, and maintain an enterprise AI gateway to centralize access, governance, and monitoring of AI model consumption.
- Develop and implement intelligent routing techniques across multiple large language models (LLMs) and AI providers.
- Evaluate and integrate local/on-premises models as alternatives to third-party hosted models to reduce operational costs.
- Establish frameworks for measuring and demonstrating cost savings through model selection, routing optimization, and infrastructure decisions.
- Provide technical leadership and mentorship to engineering teams adopting Generative AI capabilities and a team of AI/ML engineers.
What you'll need
- Demonstrated experience architecting Generative AI solutions at an enterprise scale.
- Experience researching and integrating LLMs and autonomous AI agent architecture into development processes.
- Ability to build and optimize agentic AI systems that can interact with APIs, databases, and development environments (such as LangChain, OpenAI APIs).
- Experience fine-tuning LLMs (GPT, Llama, Mistral, Claude, Gemini) for domain-specific applications.
- Proficiency in optimizing models for local inference through quantization, pruning, and distillation.
- Experience deploying models on-prem or at the edge using frameworks such as PyTorch, TensorRT, ONNX, vLLM, or llama.cpp.
- Ability to build and maintain training and inference pipelines for reproducibility and scalability.
- Hands-on experience designing and implementing AI gateway solutions and model routing techniques.
- Master's or Ph.D. in Computer Science, AI, Machine Learning, or a related field.
- 5+ years of experience in AI and machine learning, with at least 2 years of experience working on LLMs, code generation, RAG, or AI-powered automation.
Skills: LLMs, PyTorch, TensorRT, quantization, inference pipelines, edge deployment
This role has been open 27 days — well below the 54-day median for Infrastructure/Platform Software roles.
Infrastructure/Platform Software · AI ML Hardware
|
Open roles in category
101
|
Median days open
54 d
|
Median salary
$220k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | Lattice Semiconductor | All employers we track in this specialty (101 roles · 29 employers) |
|---|---|---|
| Open roles in this specialty | 1 | 101 |
| Open roles in the wider Software, Firmware & Systems family | 46 | 6119 · 146 employers |
| Median days open | 27 d | 54 d (−27 d vs this employer) |
| Median salary (USD postings) | — | $220k |
Skills observed across this category: LLMs, PyTorch, TensorRT, quantization, inference pipelines, edge deployment
Who's hiring in this category
- Qualcomm · 22 open roles · median 51 d
- NVIDIA · 19 open roles · median 51 d
- AMD · 9 open roles · median 47 d
- Intel Corporation · 8 open roles · median 21 d
- Micron Technology · 5 open roles · median 56 d
- NXP Semiconductors · 5 open roles · median 47 d
How we counted: 101 open Infrastructure/Platform Software (AI ML Hardware) roles from 29 employers tracked in the SemiconductorJobs index, counted 14 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.