GTC ON-DEMAND

 
SEARCH SESSIONS
SEARCH SESSIONS

Search All
 
Refine Results:
 
Year(s)

SOCIAL MEDIA

EMAIL SUBSCRIPTION

 
 

GTC ON-DEMAND

Deep Learning & AI Frameworks
Presentation
Media
Untangling Information for Training Strategies in Machine and Deep Learning
Abstract:
Training machine and deep learning architectures remains challenging and resource-demanding. Much of the difficulty arises from stochastic mini-batch sampling, which results in noisy cost functions for which well-known deterministic approaches fail. Strategies proposed to overcome these challenges include basic fixed learning rates (or sub-gradient approaches), a priori selected learning-rate schedules, learning-rate cycling, and grid search to probabilistic line searches. Our talk will untangle available information used for training such as noisy function values and noisy gradients. We'll demonstrate the training benefits of exploiting noisy gradient information in more advanced training strategies for both Tensorflow and Pytorch on NVIDIA GeForce GTX 1080Ti and RTX 2080 Ti GPUs. We'll show a simulated dataset of the comminution process of an industrial ball mill, which we generated using a multi-GPU discrete element code.
 
Topics:
Deep Learning & AI Frameworks, Algorithms & Numerical Techniques
Type:
Talk
Event:
GTC Silicon Valley
Year:
2019
Session ID:
S9647
Streaming:
Download:
Share: