GTC ON-DEMAND

 
SEARCH SESSIONS
SEARCH SESSIONS

Search All
 
Refine Results:
 
Year(s)

SOCIAL MEDIA

EMAIL SUBSCRIPTION

 
 

GTC ON-DEMAND

Presentation
Media
Abstract:
A variety of deep learning frameworks now make it simple to train deep neural networks of many types. However, scaling deep learning frameworks to large models with data parallel training on many GPUs remains a challenge, as the default utilities for inter-device and inter-node communication provided by these frameworks are often not optimal. Using examples from several frameworks, we demonstrate that linear strong scaling to many nodes and many devices can be achieved augmenting deep learning frameworks with CUDA-aware MPI allreduce and allgather operations, which allow them to be used in an HPC setting where multi-GPU nodes are augmented with high-speed Infiniband interconnects. We'll show that these operations allow us to quickly train very large speech recognition models.
A variety of deep learning frameworks now make it simple to train deep neural networks of many types. However, scaling deep learning frameworks to large models with data parallel training on many GPUs remains a challenge, as the default utilities for inter-device and inter-node communication provided by these frameworks are often not optimal. Using examples from several frameworks, we demonstrate that linear strong scaling to many nodes and many devices can be achieved augmenting deep learning frameworks with CUDA-aware MPI allreduce and allgather operations, which allow them to be used in an HPC setting where multi-GPU nodes are augmented with high-speed Infiniband interconnects. We'll show that these operations allow us to quickly train very large speech recognition models.  Back
 
Topics:
Artificial Intelligence and Deep Learning
Type:
Talk
Event:
GTC Silicon Valley
Year:
2017
Session ID:
S7543
Download:
Share:
 
Abstract:
WaveNet is a generative neural network architecture for audio in the time domain. Due to the high sampling frequency of audio signals and the sequential dependencies between timesteps, inference in a WaveNet model is incredibly expensive, and can take many minutes to generate a single second of audio with an unoptimized implementation. We implement custom WaveNet inference kernels and demonstrate that an efficient implementation on a CPU or a GPU can provide faster than realtime audio generation, even though neither platform is perfectly suited to such a task due to the effective lack of parallelism and high compute requirements. To our knowledge, this is the first demonstration that neural audio generation can be done efficiently enough to deploy in a production text-to-speech system.
WaveNet is a generative neural network architecture for audio in the time domain. Due to the high sampling frequency of audio signals and the sequential dependencies between timesteps, inference in a WaveNet model is incredibly expensive, and can take many minutes to generate a single second of audio with an unoptimized implementation. We implement custom WaveNet inference kernels and demonstrate that an efficient implementation on a CPU or a GPU can provide faster than realtime audio generation, even though neither platform is perfectly suited to such a task due to the effective lack of parallelism and high compute requirements. To our knowledge, this is the first demonstration that neural audio generation can be done efficiently enough to deploy in a production text-to-speech system.  Back
 
Topics:
Artificial Intelligence and Deep Learning, Tools & Libraries, Signal and Audio Processing
Type:
Talk
Event:
GTC Silicon Valley
Year:
2017
Session ID:
S7544
Download:
Share:
 
 
Previous
  • Amazon Web Services
  • IBM
  • Cisco
  • Dell EMC
  • Hewlett Packard Enterprise
  • Inspur
  • Lenovo
  • SenseTime
  • Supermicro Computers
  • Synnex
  • Autodesk
  • HP
  • Linear Technology
  • MSI Computer Corp.
  • OPTIS
  • PNY
  • SK Hynix
  • vmware
  • Abaco Systems
  • Acceleware Ltd.
  • ASUSTeK COMPUTER INC
  • Cray Inc.
  • Exxact Corporation
  • Flanders - Belgium
  • Google Cloud
  • HTC VIVE
  • Liqid
  • MapD
  • Penguin Computing
  • SAP
  • Sugon
  • Twitter
Next