Machine Learning Engineer - New Grad

Optimize speculative decoding models and LLM inference for SambaNova's Reconfigurable Dataflow Unit (RDU).

Originally posted 3 November 2026 by the employer.

About the role

As a new grad Machine Learning Engineer, you will train and evaluate models that make large language models cheaper and more capable to serve. You will focus on model-side optimizations such as speculative decoding, quantization, and agentic workflows targeting SambaNova's Reconfigurable Dataflow Unit (RDU) architecture.

What you'll do

• Train, evaluate, and tune speculative decoding models, including Dspark/Dflash, Medusa/Eagle-style heads, and multi-token prediction, to increase acceptance rates on production workloads.
• Develop and implement optimization techniques, including quantization, sparsity, distillation, pruning, and parameter-efficient fine-tuning, to quantify trade-offs between model quality, latency, and resource efficiency.
• Build and evaluate agentic systems involving tool calling, multi-step planning, retrieval, and long-context reasoning on top of the serving stack.
• Design rigorous evaluations and benchmarks to substantiate accuracy and quality claims.
• Translate recent research into working implementations and run ablations to isolate performance drivers.
• Partner with AI performance, compiler, runtime, and product teams to integrate model improvements into SambaNova products.

What you'll need

• B.S. in Computer Science, Electrical Engineering, Mathematics, or a related field.
• Strong programming skills in Python.
• Solid foundation in machine learning, linear algebra, probability, and numerical computing.
• Hands-on experience with PyTorch, TensorFlow, or JAX.
• Working knowledge of transformer architectures and the LLM training or inference pipeline.
• Ability to design clean experiments, interpret results, and communicate findings.

Nice to have

• M.S. or Ph.D. in Computer Science, Electrical Engineering, or a related field.
• Research or internship experience in LLM training, post-training, or inference.
• Experience with speculative decoding, distillation, quantization, sparsity, or other model efficiency techniques.
• Experience building agentic systems, tool-calling pipelines, or LLM evaluation harnesses.
• Familiarity with reinforcement learning or preference optimization (RLHF, DPO, GRPO, reward modeling).
• Experience with distributed training frameworks (FSDP, DeepSpeed, Megatron) or inference frameworks such as vLLM, SGLang, or TensorRT-LLM.
• Publications at ML venues (NeurIPS, ICML, ICLR, ACL, EMNLP) or open source contributions.

Skills: LLM, speculative decoding, PyTorch, quantization, compiler

More semiconductor jobs: SambaNova jobs · AI chip design jobs · Austin jobs