SEARCH SESSIONS

Search All
 
Refine Results:
 
Year(s)

SOCIAL MEDIA

EMAIL SUBSCRIPTION

 
 

GTC On-Demand

Tag Filter(s): AI and DL Research
Presentation
Media
Training Neural Networks with Mixed Precision: Real Examples
Benjamin Barsdell (NVIDIA), Michael O'Connor (NVIDIA), Christian M. Sarofeen (NVIDIA)
We will cover the techniques for training DNNs with Tensor Cores described in "S8923 - Training Neural Networks with Mixed Precision: Theory and Practice". These methods were introduced for AI processing with the Volta GPU architecture. T ...Read More
We will cover the techniques for training DNNs with Tensor Cores described in "S8923 - Training Neural Networks with Mixed Precision: Theory and Practice". These methods were introduced for AI processing with the Volta GPU architecture. Tensor Cores provide up to 120 TFlops throughput, mixing operations on IEEE half- and single-precision floats. Techniques used will include loss-scaling, master weights copy, and choosing the proper precision for a given operation. For each of TensorFlow and PyTorch we will describe a fp32 network definition and then demonstrate the same network using mixed precision techniques.  Back
 
Keywords:
AI and DL Research, Algorithms and Numerical Techniques, GTC Silicon Valley 2018 - ID S81012
Streaming:
Download: