Intern, AI Application Engineer
Deploy and benchmark vLLM inference workloads on NVIDIA H200 GPU servers with NVMe-oF KV cache offloading.
Originally posted 2 November 2026 by the employer.
About the role
This role involves deploying, benchmarking, and profiling LLM inference workloads to evaluate emerging memory and storage solutions. You will configure and test KV cache offloading to network-attached storage using frameworks like vLLM across GPU server systems.
What you'll do
- Deploy and operate an LLM inference service (vLLM serving Llama-3.3-70B) on a 4 × NVIDIA H200 GPU server in collaboration with the storage architecture team.
- Build an automated benchmark harness that replays long-context, multi-turn, and agent-style request traces while recording latency and throughput metrics including TTFT, TPOT, and goodput.
- Configure KV cache offloading (LMCache, NVIDIA Dynamo/NIXL) to network-attached storage over NVMe-oF and RDMA on a BlueField-3 storage server and a network-attached SSD prototype.
- Run daily experiment sweeps across request rates, context lengths, and cache sizes, maintaining reproducibility in git.
- Analyze bottlenecks across the GPU, network, and SSD using Python, and summarize measured storage requirements (bandwidth, IOPS, latency) for future SSD controller architectures.
- Collaborate with FPGA, SoC, and firmware engineers to debug the end-to-end data path and validate KV cache correctness.
What you'll need
- Currently pursuing a Bachelors, Masters, or PhD in Computer Science, Computer Engineering, Electrical Engineering, or a related field with at least 1 academic quarter/semester remaining.
- Hands-on experience running LLM inference with Python and PyTorch on NVIDIA GPUs under Linux, using a serving framework such as vLLM, SGLang, or Hugging Face TGI.
- Understanding of LLM serving internals: KV cache, prefix caching, batching and parallelism (tensor/pipeline), and latency metrics such as TTFT, TPOT, and throughput.
- Experience benchmarking and profiling system performance, as well as analyzing results with scripts (e.g., Python, pandas, plotting) to explain bottlenecks.
Nice to have
- Familiarity with KV cache offloading (LMCache, NVIDIA Dynamo/NIXL).
- Familiarity with RDMA/RoCE or NVMe/NVMe-oF storage.
- Familiarity with containers (Docker/Kubernetes).
Skills: vLLM, PyTorch, KV cache, NVMe-oF, GPU, benchmarking
More semiconductor jobs: Samsung Semiconductor jobs · California jobs
This role has been open 0 days — well below the 52-day median for Infrastructure/Platform Software roles.
Infrastructure/Platform Software · Infrastructure Platform
|
Open roles in category
1001
|
Median days open
52 d
|
Category median salary (all employers)
$236k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | Samsung Semiconductor | All employers we track in this specialty (1001 roles · 90 employers) |
|---|---|---|
| Open roles in this specialty | 3 | 1001 |
| Open roles in the wider Software, Firmware & Systems family | 15 | 6322 · 144 employers |
| Median days open | 30 d | 52 d (−22 d vs this employer) |
| Median salary (USD postings) | — | $236k |
Skills observed across this category: vLLM, PyTorch, KV cache, NVMe-oF, GPU, benchmarking
Who's hiring in this category
- NVIDIA · 303 open roles · median 56 d
- Qualcomm · 58 open roles · median 55 d
- AMD · 49 open roles · median 32 d
- Graphcore · 49 open roles · median 108 d
- Intel Corporation · 46 open roles · median 18 d
- Arm Holdings · 36 open roles · median 5 d
How we counted: 1001 open Infrastructure/Platform Software (Infrastructure Platform) roles from 90 employers tracked in the SemiconductorJobs index, counted 2 Nov 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.