Skip to main content
Normal Computing

Runtime Engineer

Normal Computing Mountain View, CA, United States Full-time 1 month ago
Embedded & Systems Software

Build the runtime architecture, hardware abstraction layer, and device management interfaces for in-memory LLM inference hardware.

Originally posted October 20, 2026 by the employer.

About the role

As a Runtime Engineer at Normal Computing, you will own the technical direction, design, and implementation of the runtime layer that loads, schedules, and executes compiled programs on custom hardware. You will build this layer against simulators and FPGA prototypes to create a stable hardware abstraction for compiler and serving layers before silicon arrives.

What you'll do

  • Own the runtime architecture, including device management, memory allocation, host-device transfers, executable loading, and command submission.
  • Build a hardware abstraction interface that conceals the differences between simulators, FPGA prototypes, and future silicon generations.
  • Manage persistent device state across requests, including resident model weights, to support serving layers.
  • Minimize runtime overhead on the critical path for LLM inference to optimize end-to-end latency and throughput.
  • Define interface boundaries with the compiler layer above and the driver and firmware layers below.

What you'll need

  • Experience building runtimes, drivers, or hardware abstraction layers for accelerators or non-standard hardware.
  • Strong systems programming experience in C++ or Rust covering memory management, concurrency, and low-level performance.
  • Track record of owning a complex systems component end-to-end from initial design through production.
  • Working understanding of LLM inference workloads and their runtime requirements.
  • Experience developing software for pre-silicon targets across simulation and bring-up environments.

Nice to have

  • Experience with open runtime architectures such as IREE's HAL, or internal familiarity with CUDA or ROCm runtimes.
  • Experience with in-memory compute or other non-von-Neumann hardware architectures.
  • Experience integrating a custom runtime with inference serving frameworks such as vLLM.
  • Contributions to open-source runtime or systems infrastructure projects.

More semiconductor jobs: Normal Computing jobs · Embedded software jobs · Mountain View jobs

Market context

Tracked in our index since 20 Oct 2026. This is our observation date, not the employer posting date. Specialty: Systems Software Engineering (Systems Software). 720 active roles in this specialty at 79 employers in our index, as of 20 Nov 2026.

How we counted: postings read from employer career sites. How we count

Normal Computing

Normal Computing

Apply now
Mountain View, CA, United States
On-site
Full-time
1 month ago

Share this job