Skip to content
AI.info

jobs

Engineering Manager, Inference Performance

ABOUT BASETEN Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer.

Company
Baseten
Location
San Francisco; New York
Status
Open
Posted
2026-10-07 00:57 UTC

Baseten is hiring an engineering manager to lead part of its Inference Performance team, which makes demanding AI workloads run faster on GPUs. The manager hires and mentors inference performance engineers while staying close to the technical work: reviewing designs, guiding profiling, and productionizing quantization, speculative decoding, KV-cache reuse, chunked prefill and custom scheduling. Requirements include experience managing engineers, a GPU optimization background, and familiarity with PyTorch or TensorRT-LLM. The hybrid role spans San Francisco and New York; success is measured in tokens per GPU-hour, latency and cost.

Original job posting