SEARCH SESSIONS

Search All
 
Refine Results:
 
Year(s)

SOCIAL MEDIA

EMAIL SUBSCRIPTION

 
 

GTC On-Demand

Presentation
Media
Abstract:
In this session, participants will get a taste of state-of-the-art techniques for scaling Deep Learning on GPU clusters. We present SuperML, a general and efficient communication layer for machine learning, which can scale neural network training to hundreds of GPU nodes. SuperML builds on three main ideas: decentralization, which allows algorithms to converge without a centralized coordinator (parameter server) or all-to-all communication, communication quantization, which significantly speeds up point-to-point messaging, and structured sparsity, by which SuperML induces model updates which only have a limited number of non-zero entries. From the technical perspective, SuperML provides a new implementation of the classic MPI standard, re-designed and re-implemented to provide efficient support for quantization and sparsity. We illustrate the performance characteristics of SuperML on CSCS Piz Daint, Europe's most powerful supercomputer, and on Amazon EC2, improving upon other highly optimized implementations such as CrayMPI and NVIDIA NCCL.
In this session, participants will get a taste of state-of-the-art techniques for scaling Deep Learning on GPU clusters. We present SuperML, a general and efficient communication layer for machine learning, which can scale neural network training to hundreds of GPU nodes. SuperML builds on three main ideas: decentralization, which allows algorithms to converge without a centralized coordinator (parameter server) or all-to-all communication, communication quantization, which significantly speeds up point-to-point messaging, and structured sparsity, by which SuperML induces model updates which only have a limited number of non-zero entries. From the technical perspective, SuperML provides a new implementation of the classic MPI standard, re-designed and re-implemented to provide efficient support for quantization and sparsity. We illustrate the performance characteristics of SuperML on CSCS Piz Daint, Europe's most powerful supercomputer, and on Amazon EC2, improving upon other highly optimized implementations such as CrayMPI and NVIDIA NCCL.  Back
 
Topics:
AI and DL Research, Accelerated Analytics, HPC and Supercomputing
Type:
Talk
Event:
GTC Silicon Valley
Year:
2018
Session ID:
S8668
Streaming:
Download:
Share:
 
Abstract:
We'll present a suite of artificial intelligence applications and computation geared towards increasing our understanding of the universe. The intensive collaboration between astrophysics and computer science has long started since Jim Gray and Alex Szalay. Nowadays, astrophysics continues to offer rich datasets, which are ideal for exploration with the latest in AI and computer science in general. We'll present successful projects in our space.ml initiative that try to answer a range of fascinating astrophysics questions. We'll show how we can use generative adversarial networks to go slightly beyond the Nyquist resolution limit in images, and to study the host galaxies of powerful quasars. We demonstrate how we can use transfer learning to identify rare galaxy mergers, and how to use variational autoencoders to forward model the processes in cosmology and galaxy evolution. We'll illustrate how we can use GPUs for compressive sensing to better analyze data from radio arrays, and to model the evolution of black holes over the age of the universe. Attendees will not only get our current answers to these questions but also get a taste of how AI is reshaping science today.
We'll present a suite of artificial intelligence applications and computation geared towards increasing our understanding of the universe. The intensive collaboration between astrophysics and computer science has long started since Jim Gray and Alex Szalay. Nowadays, astrophysics continues to offer rich datasets, which are ideal for exploration with the latest in AI and computer science in general. We'll present successful projects in our space.ml initiative that try to answer a range of fascinating astrophysics questions. We'll show how we can use generative adversarial networks to go slightly beyond the Nyquist resolution limit in images, and to study the host galaxies of powerful quasars. We demonstrate how we can use transfer learning to identify rare galaxy mergers, and how to use variational autoencoders to forward model the processes in cosmology and galaxy evolution. We'll illustrate how we can use GPUs for compressive sensing to better analyze data from radio arrays, and to model the evolution of black holes over the age of the universe. Attendees will not only get our current answers to these questions but also get a taste of how AI is reshaping science today.  Back
 
Topics:
AI Application Deployment and Inference, Astronomy and Astrophysics
Type:
Talk
Event:
GTC Silicon Valley
Year:
2018
Session ID:
S8667
Streaming:
Download:
Share:
 
Abstract:

We'll present new techniques for training machine learning models using low-precision computation and communication. We'll start by briefly outlining new theoretical results proving that, surprisingly, many fundamental machine learning tools, such as dense generalized linear models, can be trained end-to-end (samples, model, and gradients) using low precision (as little as one bit per value), while still guaranteeing convergence. We'll then explore the implications of these techniques with respect to two key practical applications: multi-GPU training of deep neural networks, and compressed sensing for medical and astronomical data.

We'll present new techniques for training machine learning models using low-precision computation and communication. We'll start by briefly outlining new theoretical results proving that, surprisingly, many fundamental machine learning tools, such as dense generalized linear models, can be trained end-to-end (samples, model, and gradients) using low precision (as little as one bit per value), while still guaranteeing convergence. We'll then explore the implications of these techniques with respect to two key practical applications: multi-GPU training of deep neural networks, and compressed sensing for medical and astronomical data.

  Back
 
Topics:
Deep Learning and AI, Performance Optimization
Type:
Talk
Event:
GTC Silicon Valley
Year:
2017
Session ID:
S7580
Download:
Share: