Skip to main content
ElastixAI

AI Software Engineer

ElastixAI Seattle, United States Full-time 9 months ago
Embedded & Systems Software

Design low-level serving stacks and integrate custom accelerator hardware using vLLM, SGLang, and TensorRT-LLM.

Originally posted February 19, 2026 by the employer.

About the role

As an AI Software Engineer on the core inference platform team, you will design and extend the low-level serving stack across ML software and custom accelerator hardware. The work centers on extending open-source frameworks such as vLLM, SGLang, and TensorRT-LLM, developing model sharding and scheduling logic, and integrating with a proprietary AI accelerator.

What you'll do

  • Architect, extend, and optimize core components of the AI serving platform for throughput, latency, and scalability.
  • Customize open-source serving frameworks such as vLLM for proprietary model ingestion and accelerator integration.
  • Develop efficient model partitioning, scheduling, and memory management strategies for multi-device inference.
  • Collaborate with ML engineers on model export and runtime optimization, including quantization and graph transforms.
  • Work with hardware engineers to influence accelerator interface design and performance tuning.
  • Profile, debug, and optimize across the full stack, from Python orchestration to C++ kernels and PCIe drivers.

What you'll need

  • BS, MS, or PhD in Computer Science, Electrical/Computer Engineering, or a related field.
  • 3+ years of professional experience in systems programming, ML infrastructure, or distributed inference.
  • Proficiency in C++ and Python, with strong debugging and performance analysis skills.
  • Deep familiarity with one or more LLM serving frameworks, such as vLLM, SGLang, TensorRT-LLM, or DeepSpeed-Inference.
  • Understanding of model deployment internals, including token scheduling, KV caching, batching, and pipelined inference.
  • Experience working close to the hardware abstraction layer, including CUDA, PCIe, memory management, or runtime scheduling.

Nice to have

  • Experience with hardware-aware ML optimization, compiler/runtime integration, or accelerator SDKs.
  • Hands-on experience profiling GPU/accelerator workloads.
  • Familiarity with containerized deployments using Docker or Kubernetes.
  • Exposure to distributed systems or large-scale inference clusters.
  • Contributions to open-source ML or serving frameworks.

Skills: vLLM, C++, accelerator integration, runtime, CUDA

Market context

First seen in our index 2 days ago, well below the 59-day median for Infrastructure/Platform Software roles.

Infrastructure/Platform Software · Infrastructure Platform

Open roles in category
1060
Median days since first seen
59 d
Category median salary (all employers)
$236k
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
How ElastixAI compares in Infrastructure/Platform Software hiring
Metric ElastixAI All employers we track in this specialty (1060 roles · 98 employers)
Open roles in this specialty 2 1060
Open roles in the wider Software, Firmware & Systems family 3 6787 · 180 employers
Median days since first seen 2 d 59 d (+57 d vs this employer)
Median salary (USD postings) n/a $236k

Skills observed across this category: vLLM, C++, accelerator integration, runtime, CUDA

Who's hiring in this category

  • NVIDIA · 359 open roles · median 61 d
  • Qualcomm · 69 open roles · median 60 d
  • AMD · 56 open roles · median 40 d
  • Graphcore · 47 open roles · median 122 d
  • Arm Holdings · 38 open roles · median 19 d
  • Cerebras · 36 open roles · median 111 d

How we counted: 1060 open Infrastructure/Platform Software (Infrastructure Platform) roles from 98 employers tracked in the SemiconductorJobs index, counted 16 Nov 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there, and is counted in the wider Software, Firmware & Systems family row. Days are counted from when our index first saw each posting, not from the employer's original posting date.

Apply now
Seattle, United States
On-site
Full-time
9 months ago

Share this job