ML Systems Engineer, Large-Scale Model Training & RL Infrastructure

The role 

Nebius Token Factory is building an AI training and model post-training capability for frontier model improvement. This role owns the infrastructure that makes large-scale training and RL experiments possible, reliable, reproducible, and efficient. The work sits at the intersection of distributed systems, GPU performance, model training frameworks, RL pipelines, and production engineering.

A Senior ML Systems Engineer owns substantial training or RL infrastructure components end to end. They are deeply hands-on, can debug difficult distributed training failures independently, and can deliver measurable improvements in experiment throughput, stability, and GPU utilization.

Your responsibilities: 

• Build and maintain distributed training infrastructure for SFT, continued pretraining, preference optimization, and RL workloads.

• Integrate and extend frameworks such as Megatron-LM, DeepSpeed, PyTorch FSDP/DTensor, Ray, verl, slime, AReaL, OpenRLHF, or equivalent internal systems.

• Implement and debug parallelism strategies including tensor, pipeline, sequence/context, expert, and data parallelism.

• Build reliable rollout, reward model serving, replay/data buffer, checkpointing, evaluation, and experiment orchestration components for RL training.

• Profile and improve GPU utilization, communication efficiency, memory usage, and training throughput.

• Diagnose failures across NCCL, CUDA, PyTorch, Ray, schedulers, storage, networking, and checkpointing layers.

• Create reproducible training runs, launch scripts, dashboards, runbooks, and operational tooling for research users.

• Partner with research scientists to turn algorithmic training recipes into scalable, debuggable systems.

• Write clear design docs, incident reports, benchmark reports, and operating guides.

Must-haves: 

• Strong Python and PyTorch engineering skills.

• Hands-on experience with distributed model training, large-scale ML systems, or GPU cluster workloads.

• Practical understanding of transformer training bottlenecks, memory pressure, gradient/optimizer state, communication overhead, and checkpointing.

• Experience debugging production or research training jobs across multiple GPUs or nodes.

• Ability to reason quantitatively about throughput, utilization, memory, reliability, cost, and research velocity.

• Strong communication skills and ability to collaborate with researchers, ML engineers, platform engineers, and leadership.

Nice-to-haves: 

• Experience with Megatron-LM, DeepSpeed, PyTorch FSDP/DTensor, Ray, Slurm, Kubernetes, or large internal training platforms.

• Experience with RL infrastructure frameworks such as verl, slime, AReaL, OpenRLHF, TRL, or custom PPO/GRPO/RLHF systems.

• Familiarity with NCCL, CUDA, Triton, Nsight, InfiniBand, RDMA, RoCE, H100/H200/B200 clusters, or storage/network bottlenecks.

• Experience supporting SFT, DPO, PPO, GRPO, RLAIF, reward model serving, rollout generation, or agent training workloads.

• Open-source contributions to distributed training, RL infrastructure, PyTorch, Ray, Megatron, DeepSpeed, or related systems.

Key employee benefits in the US:

• Health insurance: 100% company-paid medical, dental, and vision coverage for employees and families.

• 401(k) plan: Up to 4% company match with immediate vesting.

• Parental leave: 20 weeks paid for primary caregivers, 12 weeks for secondary caregivers.

• Remote work reimbursement: Up to $85/month for mobile and internet.

• Disability & life insurance: Company-paid short-term, long-term and life insurance coverage.