The vision for the PyTorch Distributed team is to make PyTorch the industry leading machine learning framework for Efficient and Large Scale Distributed Training and Inference. This vision includes: - Enabling scaling to thousands of GPUs for critical workloads like generative AI and recommendation. - Work within meta and with leading industry collaborators to ship impact worldwide. - Outpace our competitors as the preferred framework across the industry for distributed training - Enable cutting edge research for the largest and most complex models - Provide easy to use high-level APIs for data, model, pipeline, hybrid parallelism and auto parallelism - Provide a rich toolkit of basic building blocks that allow exploration of a variety of distributed training and inference paradigms The team is highly innovative, passionate about the technologies we build, and loves to do deep technical work.