$184,000 - $356,500 USD yearly
NeMo Evaluator is the evaluation and observability infrastructure within NVIDIA's NeMo Platform.
Originally posted November 6, 2026 by the employer. Relisted by the employer on November 7, 2026.
About the role
This role focuses on NeMo Evaluator within the NeMo Platform, building infrastructure for developing, evaluating, deploying, and operating AI systems at scale. You will develop practical infrastructure for observing behavior, measuring progress, catching regressions, and iterating on AI agents.
What you'll do
- Design and implement Python-first APIs, SDK workflows, and plugin interfaces for building, measuring, and improving agents across multiple runtimes and product surfaces.
- Build reusable systems for observing behavior, measuring progress, detecting regressions, and turning runtime evidence into product decisions.
- Build systems for ingesting, normalizing, validating, and analyzing agent execution data and evaluation datasets.
- Partner with research, product, platform, and infrastructure teams to integrate agentic capabilities across NVIDIA agent runtimes and developer workflows.
- Improve reliability, observability, debuggability, and performance across NeMo Platform, SDKs, plugins, jobs, and developer workflows.
- Build test coverage across unit, integration, E2E, Docker, and Kubernetes workflows while providing technical leadership through design reviews, code reviews, and mentoring.
What you'll need
- BS, MS, or equivalent experience in Computer Science, Computer Engineering, or a related technical field.
- 5+ years of professional software engineering experience building production systems.
- Python engineering skills covering API design, typing, testing, debugging, performance analysis, and maintainable software design.
- Experience designing SDKs, libraries, plugins, CLIs, or other developer-facing interfaces.
- Experience with distributed systems, cloud-native services, containers, Kubernetes, or job orchestration.
- Experience with structured data modeling and validation systems such as Pydantic, typed schemas, event/trace models, or SDK-generated types.
Skills: Python, SDK, API design, Distributed systems, Kubernetes
More semiconductor jobs: NVIDIA jobs
Tracked in our index since 29 May 2026. This is our observation date, not the employer posting date. Specialty: Infrastructure/Platform Software (Infrastructure Platform). 1064 active roles in this specialty at 98 employers in our index, as of 19 Nov 2026.
How we counted: postings read from employer career sites. How we count