jobs

Senior AI R&D Engineer - Performance - Pre-training (f/m/d)

Aleph Alpha is hiring a Senior AI R&D Engineer specializing in performance for its pre-training organization in Heidelberg. The role is focused on maximizing hardware utilization and training throughput for foundation models running on larg

Company

Aleph Alpha

Location

Heidelberg, Germany

Status

Closed

Posted

2026-09-19T03:35:30.059+00:00

Aleph Alpha's pre-training group in Heidelberg needs a senior engineer to make foundation-model training faster on large GPU clusters. The work spans deep-learning frameworks, distributed systems and GPU architecture: profiling training loops with PyTorch Profiler and Nsight tools, finding kernel- and system-level bottlenecks, tuning tensor, data, expert and fully sharded parallelism, and improving communication-to-computation trade-offs. The posting asks for Python and PyTorch fluency, parallel or distributed systems experience, and CUDA, NCCL or MPI knowledge. Contributing to TorchTitan, Megatron-LM or DeepSpeed is a plus. Joining the Training Efficiency Team involves regular on-site collaboration.