GTC ON-DEMAND

 
SEARCH SESSIONS
SEARCH SESSIONS

Search All
 
Refine Results:
 
Year(s)

SOCIAL MEDIA

EMAIL SUBSCRIPTION

 
 

GTC ON-DEMAND

Presentation
Media
Abstract:
Myia is an experimental deep learning framework that aims to help researchers implement differentiable models with complex control flow and run them efficiently on GPUs. These models may be implemented using a subset of Python that includes conditionals, loops, dataclasses, higher order functions, as well as recursion. Myia is then able to run powerful type and shape inference (without the need for type annotations) to guarantee correctness prior to running the model, differentiate the whole or parts of the model, and finally aggressively optimize the resulting code to execute it as fast as possible. Unlike other frameworks such as TensorFlow or PyTorch, which are more like domain-specific languages or libraries, our approach is grounded in programming language and compiler theory, which affords it greater generality. We demonstrate Myia on a variety of models requiring complex control flow and contrast program size and performance with other frameworks.
Myia is an experimental deep learning framework that aims to help researchers implement differentiable models with complex control flow and run them efficiently on GPUs. These models may be implemented using a subset of Python that includes conditionals, loops, dataclasses, higher order functions, as well as recursion. Myia is then able to run powerful type and shape inference (without the need for type annotations) to guarantee correctness prior to running the model, differentiate the whole or parts of the model, and finally aggressively optimize the resulting code to execute it as fast as possible. Unlike other frameworks such as TensorFlow or PyTorch, which are more like domain-specific languages or libraries, our approach is grounded in programming language and compiler theory, which affords it greater generality. We demonstrate Myia on a variety of models requiring complex control flow and contrast program size and performance with other frameworks.  Back
 
Topics:
Deep Learning & AI Frameworks
Type:
Talk
Event:
GTC Silicon Valley
Year:
2019
Session ID:
S9244
Streaming:
Download:
Share:
 
Abstract:
Myia is a new, experimental deep learning framework that aims to provide to deep learning researchers both the expressive power and the performance that they need. Symbolic frameworks such as TensorFlow only cover a curated subset of programming language features and do not support second order gradients very well. Dynamic frameworks such as PyTorch, while very powerful, use an operator overloading approach for automatic differentiation, which does not lend itself well to optimization. With Myia, we attempt to have the best of both worlds: we implement a general and composable approach to automatic differentiation over a functional abstraction of a subset of the Python programming language. That subset includes if, while, for, and recursion, providing plenty of expressive power, and yet it can also be analyzed statically to provide the best possible performance. We''ll present the Myia language from a high-level technical perspective, including a short primer on functional programming and automatic differentiation. It is of special interest to deep learning framework or library implementers.
Myia is a new, experimental deep learning framework that aims to provide to deep learning researchers both the expressive power and the performance that they need. Symbolic frameworks such as TensorFlow only cover a curated subset of programming language features and do not support second order gradients very well. Dynamic frameworks such as PyTorch, while very powerful, use an operator overloading approach for automatic differentiation, which does not lend itself well to optimization. With Myia, we attempt to have the best of both worlds: we implement a general and composable approach to automatic differentiation over a functional abstraction of a subset of the Python programming language. That subset includes if, while, for, and recursion, providing plenty of expressive power, and yet it can also be analyzed statically to provide the best possible performance. We''ll present the Myia language from a high-level technical perspective, including a short primer on functional programming and automatic differentiation. It is of special interest to deep learning framework or library implementers.  Back
 
Topics:
Deep Learning & AI Frameworks, Programming Languages
Type:
Talk
Event:
GTC Silicon Valley
Year:
2018
Session ID:
S8441
Streaming:
Download:
Share:
 
 
Previous
  • Amazon Web Services
  • IBM
  • Cisco
  • Dell EMC
  • Hewlett Packard Enterprise
  • Inspur
  • Lenovo
  • SenseTime
  • Supermicro Computers
  • Synnex
  • Autodesk
  • HP
  • Linear Technology
  • MSI Computer Corp.
  • OPTIS
  • PNY
  • SK Hynix
  • vmware
  • Abaco Systems
  • Acceleware Ltd.
  • ASUSTeK COMPUTER INC
  • Cray Inc.
  • Exxact Corporation
  • Flanders - Belgium
  • Google Cloud
  • HTC VIVE
  • Liqid
  • MapD
  • Penguin Computing
  • SAP
  • Sugon
  • Twitter
Next