Design, orchestrate, and optimize large-scale LLM pre-training across 1,000+ GPUs. Implement 3D parallelism, manage GPU clusters (SLURM/Kubernetes), optimize InfiniBand/RDMA networking and memory, and automate checkpointing and failure recovery for long training runs.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
.png)
Healthtech • Social Impact • Telehealth
Provides responsive support to a remote network of therapists and clinicians through email and phone. Responsibilities include troubleshooting technical and logistical issues, answering caseload and provider-patient matching questions, tracking requests through resolution, escalating issues appropriately, and proactively building provider relationships. The role requires strong organization, follow-through, sound judgment, empathy, and comfort creating structure in an evolving startup environment.
Top Skills:
Slack
Cybersecurity
The Senior Manager, Implementation and Onboarding at 1Password leads customer onboarding, technical implementation, and team strategy, focusing on enhancing customer success and securing relationships post-sale.
Top Skills:
AdAPIsAzure AdCli ToolsDirectory ServicesOktaSaas ArchitecturesScimSso
.png)
eCommerce • Fintech • Hardware • Payments • Software • Financial Services
Own Square Hardware sales and revenue across major U.S. retail partners. Build and scale retail partnership strategies, manage existing retailer and distributor relationships, source and negotiate new partnerships, lead cross-functional launches and forecasting, and manage retail marketing budgets through test-and-learn investments.
What you need to know about the Vancouver Tech Scene
Raincouver, Vancity, The Big Smoke — Vancouver is known by many names, and in recent years, it has gained a reputation as a growing hub for both tech and sustainability. Renowned for its natural beauty, the city has become a magnet for professionals eager to create environmental solutions, and with an emphasis on clean technology, renewable energy and environmental innovation, it's attracted companies across various industries, all working toward a shared goal: advancing clean technology.
