Hyphen Connect Limited
LLM Pre-training & Distributed Engineer (AI Infrastructure)
Be an Early Applicant
Design, orchestrate, and optimize large-scale LLM pre-training across 1,000+ GPUs. Implement 3D parallelism, manage GPU clusters (SLURM/Kubernetes), optimize InfiniBand/RDMA networking and memory, and automate checkpointing and failure recovery for long training runs.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
Cloud • Fintech • Food • Information Technology • Software • Hospitality
As a Staff Software Engineer, you'll lead a team in developing and delivering scalable software solutions for employee management in the restaurant industry, focusing on enhancing customer and employee experiences.
Top Skills:
GraphQLJavaKotlinReactRestTypescript
Artificial Intelligence • Big Data • Cloud • Information Technology • Software • Cybersecurity • Data Privacy
As a Strategic Sales Engineer, you'll provide technical direction, support sales efforts, and develop integrated solutions addressing customer needs to exceed sales quotas.
Top Skills:
Backup And Disaster RecoveryCloud Data ManagementData AnalyticsSan (Storage Area Network) Systems
Automotive • Hardware • Robotics • Software • Transportation • Manufacturing
The CNC Technician develops and troubleshoots CNC programs, sets up machines, coordinates with teams, trains staff, and ensures quality production.
Top Skills:
Cnc Machine ControlsCnc ProgrammingHand ToolsMeasuring EquipmentSpc
What you need to know about the Vancouver Tech Scene
Raincouver, Vancity, The Big Smoke — Vancouver is known by many names, and in recent years, it has gained a reputation as a growing hub for both tech and sustainability. Renowned for its natural beauty, the city has become a magnet for professionals eager to create environmental solutions, and with an emphasis on clean technology, renewable energy and environmental innovation, it's attracted companies across various industries, all working toward a shared goal: advancing clean technology.
