jobs
Product Manager, Inference Platform
ABOUT BASETEN Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer.
- Company
- Baseten
- Location
- San Francisco; New York
- Status
- Open
- Posted
- 2026-11-11 18:13 UTC
Baseten is hiring a technical product manager for its inference platform, working with inference engineers and researchers to turn serving advances — batching, caching, quantization, speculative decoding, serving architectures — into features customers can configure and adopt. The job involves defining benchmarks for latency, throughput, quality and cost, studying customer workloads to prioritize optimizations, and helping customers weigh hardware, model configuration, autoscaling and routing tradeoffs. It asks for shipped AI/ML products or infrastructure, a software, ML or research background, and familiarity with GPU economics and performance measurement. Hands-on experience with vLLM, TensorRT-LLM or SGLang is listed as a plus.
Original job posting