SEARCH SESSIONS
SEARCH SESSIONS

Search All
Refine Results:
 
Year(s)

SOCIAL MEDIA

EMAIL SUBSCRIPTION

 
 

GTC ON-DEMAND

Topic(s) Filter: AI & Deep Learning Research
Presentation
Media
Abstract:
We'll cover the theory and practice for training DNNs with Tensor Cores, introduced for AI processing with the Volta GPU architecture. Tensor Cores provide up to 120 TFlops throughput, mixing operations on IEEE half- and single-precision floats. In ...Read More
Abstract:
We'll cover the theory and practice for training DNNs with Tensor Cores, introduced for AI processing with the Volta GPU architecture. Tensor Cores provide up to 120 TFlops throughput, mixing operations on IEEE half- and single-precision floats. In the theory portion of the talk, we'll review the half-precision format, values that arise in DNN computations, and techniques that maximize utilization of fp16 format by these values. Techniques include loss-scaling, master weights, and choosing the proper precision for a given operation. In the practice portion of this talk, we'll survey various models that have been trained in mixed precision, matching the accuracy of fp32 training sessions while using the same hyperparameters. Models include various architectures (feed forward, recurrent, generative) as well as cover diverse tasks (image, speech, and language processing). We'll also provide network design and training guidelines to maximize speed when using Tensor Cores.  Back
 
Topics:
AI & Deep Learning Research, Algorithms & Numerical Techniques
Type:
Talk
Event:
GTC Silicon Valley
Year:
2018
Session ID:
S8923
Streaming:
Download:
Share:
 
 
Previous
  • Amazon Web Services
  • IBM
  • Cisco
  • Dell EMC
  • Hewlett Packard Enterprise
  • Inspur
  • Lenovo
  • SenseTime
  • Supermicro Computers
  • Synnex
  • Autodesk
  • HP
  • Linear Technology
  • MSI Computer Corp.
  • OPTIS
  • PNY
  • SK Hynix
  • vmware
  • Abaco Systems
  • Acceleware Ltd.
  • ASUSTeK COMPUTER INC
  • Cray Inc.
  • Exxact Corporation
  • Flanders - Belgium
  • Google Cloud
  • HTC VIVE
  • Liqid
  • MapD
  • Penguin Computing
  • SAP
  • Sugon
  • Twitter
Next