Skip to main content
NVIDIA

Senior Deep Learning Software Engineer, Inference and Model Optimization

NVIDIA Santa Clara, CA, United States 1 month ago
Embedded & Systems Software

$184,000 - $356,500 USD yearly

Develop and scale automated inference and deployment solutions for generative AI models like LLMs and diffusion models.

Originally posted October 9, 2026 by the employer.

About the role

As a Senior Deep Learning Software Engineer, you will develop and scale automated inference and deployment solutions for generative AI models. This role involves optimizing generative AI models such as large language models (LLM) and diffusion models for maximal inference efficiency using techniques ranging from neural architecture search and pruning to sparsity, quantization, and automated deployment strategies.

What you'll do

  • Leverage the torch 2.0 ecosystem (TorchDynamo, torch.export, torch.compile) to analyze and extract standardized model graph representation.
  • Develop high-performance optimization techniques for inference, such as automated model sharding (e.g., tensor parallelism, sequence parallelism) and efficient attention kernels with kv-caching.
  • Collaborate with teams across NVIDIA to use performant kernel implementations within the automated deployment solution.
  • Analyze and profile GPU kernel-level performance to identify hardware and software optimization opportunities.
  • Architect and design a modular and scalable software platform to provide an excellent user experience with broad model support.

What you'll need

  • Masters, PhD, or equivalent experience in Computer Science, AI, Applied Math, or related field.
  • 8+ years of relevant work or research experience in Deep Learning.
  • Excellent software design skills, including debugging, performance analysis, and test design.
  • Strong proficiency in Python, PyTorch, and related ML tools (e.g., HuggingFace).
  • Strong algorithms and programming fundamentals.
  • Good written and verbal communication skills.

Skills: generative AI, LLMs, inference efficiency, PyTorch, CUDA

Market context
  • Employer posting date: 9 Oct 2026
  • First seen in our index: 10 Oct 2026 (our observation date)
  • Last confirmed on the employer's career site: 10 Nov 2026
  • Specialty: Infrastructure/Platform Software (GPU Software Stack)
  • Comparable active jobs: 345 at 25 employers, as of 20 Nov 2026
  • Related specialties (Software, Firmware & Systems family): 6895 active jobs at 180 employers

View all NVIDIA jobs

How we counted: postings read from employer career sites; dates are our observations unless labelled as the employer's. How we count

Apply now
Santa Clara, CA, United States
On-site
$184,000 - $356,500 USD yearly
1 month ago

Share this job