Senior Deep Learning Scientist, Multimodal Agentic RL - CA, Santa Clara, United States
$184,000 - $287,500 USD yearly
Originally posted 2 October 2026 by the employer.
Define core algorithmic improvements for multimodal foundation models, scaling your ideas through Nemotron Omni and VoiceChat platforms.
About the role
This role is for a Senior Deep Learning Scientist to advance efforts in streaming and agentic multimodal AI. You will develop models capable of reasoning, planning, and acting across diverse modalities.
What you'll do
- Apply fundamental and applied research to develop, train, fine-tune, and deploy large language models for agentic systems encompassing audio-visual reasoning, tool usage, and document understanding.
- Advance post-training and alignment methods including instruction tuning, preference optimization, and RLHF/RLVR/MOPD to improve multimodal agents for complex use cases.
- Research and develop agentic reasoning and grounded perception capabilities, focusing on planning, tool execution, and long-horizon task completion across digital and physical environments.
- Lead the collection, development, and benchmarking of multimodal datasets, ensuring high-quality evaluation of model accuracy, safety, and task completion success.
What you'll need
- Master’s degree or PhD in Computer Science, AI, or Applied Math with 5+ years of relevant work experience.
- Excellent programming skills in Python with strong fundamentals in scalable model development and deep learning frameworks like PyTorch.
- Strong knowledge of ML/DL techniques and modern foundation model architectures, including Transformers and mixture-of-experts models.
- Foundational understanding of reinforcement learning algorithms and implementation, including MDPs, policies, and reward design.
- Hands-on experience in post-training multimodal models for omni-modality (audio-visual) reasoning, full-duplex voice chat, and human-AI interaction.
- Proven ability to manage model development life cycles, including dataset versioning, experiment tracking, and evaluation pipelines.
Skills: deep learning, multimodal AI, reinforcement learning, foundation models, agentic systems
This role has been open 2 days — well below the 75-day median for AI/ML Hardware Engineering roles.
AI/ML Hardware Engineering · AI ML Hardware
|
Open roles in category
501
|
Median days open
75 d
|
Median salary
$232k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | NVIDIA | All employers we track in this specialty (501 roles · 75 employers) |
|---|---|---|
| Open roles in this specialty | 84 | 501 |
| Open roles in the wider Software, Firmware & Systems family | 949 | 5938 · 145 employers |
| Median days open | 71 d | 75 d (−4 d vs this employer) |
| Median salary (USD postings) | — | $232k |
Skills observed across this category: deep learning, multimodal AI, reinforcement learning, foundation models, agentic systems
Who's hiring in this category
- Qualcomm · 99 open roles · median 108 d
- NVIDIA (this employer) · 84 open roles · median 71 d
- AMD · 42 open roles · median 65 d
- Micron Technology · 30 open roles · median 57 d
- Mobileye · 20 open roles · median 72 d
- NXP Semiconductors · 15 open roles · median 62 d
How we counted: 501 open AI/ML Hardware Engineering (AI ML Hardware) roles from 75 employers tracked in the SemiconductorJobs index, counted 5 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.