jobs

Software Engineer, Data Acquisition

Overview:The Data Acquisition team within the Foundations organization at OpenAI is responsible for all aspects of data collection to support our model training operations. Our team manages web crawling and GPTBot services and works closely

Company

OpenAI

Location

San Francisco

Status

Open

Posted

2023-09-22T01:31:37.849+00:00

OpenAI's Data Acquisition team, part of its Foundations organization, is hiring a software engineer in San Francisco for on-site, full-time work. The role covers web crawling, data ingestion and search for model training: leading engineering projects, building distributed systems that handle petabytes, designing indexing and search algorithms, maintaining storage backends, and running Kubernetes and Infrastructure-as-Code deployments. The posting asks for a BS, MS or PhD in computer science, four or more years of industry experience, and expertise in large stateful distributed systems. Crawler experience is a plus.