LLM Serving Engineer

Originally posted 12 October 2026 by the employer.

This role focuses on developing optimized software to enable AI models deployed on hardware, specifically machine learning kernels, compiler tools, or model efficiency tools.

About the role

This role involves creating and implementing machine learning techniques, frameworks, and tools to enable the efficient discovery and utilization of machine learning solutions over various technology verticals or designs. The focus is on developing optimized software to enable AI models, including machine learning kernels and compiler tools, for deployment on hardware.

What you'll do

• Extend training or runtime frameworks or model efficiency software tools with new features and optimizations.
• Architect and develop machine learning hardware (co-designed with machine learning software) for inference or training solutions.
• Develop optimized software to enable AI models deployed on hardware, such as machine learning kernels or compiler tools.
• Lead the development and application of machine learning techniques into products and AI solutions.
• Develop, adapt, or prototype machine learning solutions aligned with roadmaps for complex products and system-level features.
• Oversee and provide technical expertise to others conducting experiments to train and evaluate machine learning solutions.

What you'll need

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field and 6+ years of Hardware Engineering, Software Engineering, Systems Engineering, or related work experience; OR Master's degree in Computer Science, Engineering, Information Systems, or related field and 5+ years of relevant work experience; OR PhD in Computer Science, Engineering, Information Systems, or related field and 4+ years of relevant work experience.

Nice to have

• Master's degree in Computer Science, Engineering, Information Systems, or related field.
• 5 years of experience with Machine Learning frameworks (e.g., Tensor Flow, Caffe, Caffe 2, Pytorch, Keras).
• 5 years of experience with low level interactions between operating systems (e.g., Linux, Android, QNX) and Hardware.
• 5 years of experience in embedded system development and optimization with application to a specific problem domain in ML (e.g., NLP, multi-media).
• 5 years of experience with one or more programming language suitable for machine learning (e.g., Python, R, C, C++).
• 5 years of experience using statistics and probability (e.g., conditional probability, Bayes rule).
• Developed 1 novel Machine Learning architecture(s).

Skills: Machine Learning hardware, AI models, ML kernels, compiler tools, hardware teams, LLMs