SEARCH SESSIONS

Search All
 
Refine Results:
 
Year(s)

SOCIAL MEDIA

EMAIL SUBSCRIPTION

 
 

GTC On-Demand

Presentation
Media
Abstract:
Learn how to achieve real-world speedup of neural networks using structural sparsity. Structural sparsity reduces the number of weights and computations in a way that's suitable for hardware acceleration. Over-parameterized neural networks waste memory and energy. Techniques like pruning or factorization can alleviate this during inference but they often increase training time, and achieving real-world speedups remains difficult. We'll explain how biology-inspired techniques can reduce the number of weights from quadratic to linear in the number of neurons. Compared to fully connected neural networks, these structurally sparse neural networks achieve large speedups during both training and inference, while maintaining or even improving model accuracy. We'll discuss hardware considerations and results for feed-forward and recurrent networks.
Learn how to achieve real-world speedup of neural networks using structural sparsity. Structural sparsity reduces the number of weights and computations in a way that's suitable for hardware acceleration. Over-parameterized neural networks waste memory and energy. Techniques like pruning or factorization can alleviate this during inference but they often increase training time, and achieving real-world speedups remains difficult. We'll explain how biology-inspired techniques can reduce the number of weights from quadratic to linear in the number of neurons. Compared to fully connected neural networks, these structurally sparse neural networks achieve large speedups during both training and inference, while maintaining or even improving model accuracy. We'll discuss hardware considerations and results for feed-forward and recurrent networks.  Back
 
Topics:
AI and DL Research, Algorithms and Numerical Techniques
Type:
Tutorial
Event:
GTC Silicon Valley
Year:
2019
Session ID:
S9389
Streaming:
Download:
Share:
 
Abstract:
We'll present a novel framework to combine multiple layers and modalities of deep neural networks for video classification, which is fundamental to intelligent video analytics, including automatic categorizing, searching, indexing, segmentation, and retrieval of videos. We'll first propose a multilayer strategy to simultaneously capture a variety of levels of abstraction and invariance in a network, where the convolutional and fully connected layers are effectively represented by the proposed feature aggregation methods. We'll further introduce a multimodal scheme that includes four highly complementary modalities to extract diverse static and dynamic cues at multiple temporal scales. In particular, for modeling the long-term temporal information, we propose a new structure, FC-RNN, to effectively transform the pre-trained fully connected layers into recurrent layers. A robust boosting model is then introduced to optimize the fusion of multiple layers and modalities in a unified way. In the extensive experiments, we achieve state-of-the-art results on benchmark datasets.
We'll present a novel framework to combine multiple layers and modalities of deep neural networks for video classification, which is fundamental to intelligent video analytics, including automatic categorizing, searching, indexing, segmentation, and retrieval of videos. We'll first propose a multilayer strategy to simultaneously capture a variety of levels of abstraction and invariance in a network, where the convolutional and fully connected layers are effectively represented by the proposed feature aggregation methods. We'll further introduce a multimodal scheme that includes four highly complementary modalities to extract diverse static and dynamic cues at multiple temporal scales. In particular, for modeling the long-term temporal information, we propose a new structure, FC-RNN, to effectively transform the pre-trained fully connected layers into recurrent layers. A robust boosting model is then introduced to optimize the fusion of multiple layers and modalities in a unified way. In the extensive experiments, we achieve state-of-the-art results on benchmark datasets.  Back
 
Topics:
Media and Entertainment, AI for In-Vehicle Applications, Intelligent Video Analytics and Smart Cities, Deep Learning and AI
Type:
Talk
Event:
GTC Silicon Valley
Year:
2017
Session ID:
S7497
Download:
Share: