Senior/Staff Applied Scientist, Numerical Optimization & Quantization - Austin, Texas, United States

Originally posted 21 August 2026 by the employer.

This role focuses on post-training quantization methods for large language models (LLMs) and other ML applications for optical inference engines.

About the role

This role involves developing advanced post-training quantization methods for large language models (LLMs), diffusion models, and other ML applications.

What you'll do

• Develop and execute hardware-aware post-training methods for full model quantization.
• Investigate preconditioning and formulate quantization as non-convex, discrete, constrained, or second-order optimization.
• Design controlled numerical experiments to understand potential improvements and secondary effects due to analog processing hardware.
• Build research-quality implementations and reproducible experiment harnesses for testing candidate methods.
• Adapt models from open-source repositories and customer private models, including PyTorch, Triton, JAX, and emerging frameworks.
• Design and execute re-quantization, retraining, and other model adaptation techniques to minimize accuracy loss during precision reduction.

What you'll need

• PhD, or equivalent research experience, in machine learning, applied mathematics, optimization, numerical analysis, or computer science.
• 5+ years of experience in machine learning, with at least 3 years focused on model optimization and deployment.
• Research or advanced engineering experience in neural network quantization, model compression, numerical optimization, or efficient inference.
• Strong knowledge of numerical linear algebra, including matrix factorizations, conditioning, covariance estimation, and iterative methods.
• Experience with one or more of non-convex optimization, discrete optimization, manifold optimization, second-order methods, or constrained optimization.
• Strong proficiency in PyTorch and familiarity with other ML frameworks, including JAX, Triton, and TensorFlow.

Nice to have

• Experience with low-precision inference optimization (INT8, FP8, or lower).
• Background in analog or optical computing architectures.
• Knowledge of in-memory computing paradigms and matrix-vector multiplication acceleration.
• Knowledge of randomized numerical linear algebra, sketching, or structured transforms.
• Publications in quantization, optimization, numerical linear algebra, model compression, or efficient ML.
• Experience with vector quantization, lattice methods, learned codebooks, or rate-distortion ideas.
• Experience with large-scale batch inference optimization.
• Familiarity with prefill versus decode optimization strategies in LLM inference.
• Experience conducting experiments on models large enough to expose scaling and generalization problems.

Skills: post-training quantization, LLMs, optical inference engines, PyTorch, model optimization