jobs

Software Engineer, Hardware Health

About the TeamThe Hardware Health and Observability team owns the end-to-end health lifecycle of OpenAI’s global compute fleet.Our mission is to maximize healthy, usable compute across accelerator vendors, generations, cloud providers, and

Company

OpenAI

Location

San Francisco

Status

Open

Posted

2026-05-11T21:03:23.567+00:00

OpenAI's Hardware Health and Observability team seeks a software engineer in San Francisco to keep its global compute fleet healthy for model training and inference. The work involves defining health signals across GPUs, CPUs, networking and platform infrastructure, building health checks that detect and remediate failures, reducing check latency, investigating hardware faults, and owning node lifecycle workflows like drain, quarantine, repair, RMA and return to service. The posting asks for 7+ years in software or infrastructure engineering, Python and shell scripting, distributed systems experience, and SQL or PromQL.