Fill out this form to RSVP your participation at the "Training Models at Scale" tutorial from Phillip Lippe.
The event will be
in-person first, we cannot guarantee that a stream will be available at the moment, however, notebooks and videos of the lecture will be published (look at the
website for more information). You can add your preferred email to be kept up-to-date on this event (and only this event).
: Building C Room 0.110, Science Park 904, 1098 XH Amsterdam, Netherlands.
Abstract: This "Training Models at Scale" tutorial equips you with the knowledge to efficiently train large models. We'll explore various distributed training strategies like fully-sharded data parallelism, pipeline parallelism, and tensor parallelism, alongside single-GPU optimizations including mixed precision training and gradient checkpointing. By the end, you'll gain the skills to navigate the complexities of large-scale training.
