We'll present the functionalities of NCCL (pronounced "Nickel"), a standalone library of standard collective communication routines, such as all-gather, reduce, broadcast, etc., that have been optimized to achieve high bandwidth over GPU topologies. NCCL can be used in either single- or multi-process (for example, MPI) applications.