The rise of GPU-accelerated data science and AI has come about through a combination of open source innovation and better tooling to support reproducible workflows. However, as the diverse array of deep learning libraries continue to mature, attention is moving to other parts of the AI pipeline, including simulation, ETL, and deployment. In this talk, I'll review open source projects that address these other areas, such as Numba, for implementing custom simulations and data transformations on the GPU, and PyGDF, for GPU accelerated dataframes. I'll discuss how the Anaconda Distribution and its conda packaging system helps data scientists create reproducible environments and deploy models. Finally, I'll talk about how Anaconda Enterprise allows data science teams to collaborate efficiently on GPU-accelerated projects with each other, and supports AI workflows from data exploration all the way to deployment.
Many data scientists use Anaconda and Python to increase their productivity, but don't realize they can leverage these technologies for scalable analysis. We'll survey the landscape of Python tools that empower data scientists to take their work to the next level, harnessing the growing computing capability of GPUs and clusters. We'll show the power of Python to drive distributed computation with Spark and Dask, execute large-scale machine learning with TensorFlow, and visualize large datasets right in the web browser.
This talk will describe the design and development of Chroma, a Python package for fast Monte Carlo simulation of individual optical photons propagating through particle physics experiments. Chroma implements standard ray-tracing techniques with Python and PyCUDA to provide a versatile, fast, and physically-accurate optical model that is more than 100x faster at photon propagation than the standard particle physics simulation package, GEANT4. Chroma was initially developed by a small academic team of only two people and will discuss lessons learned in the development process and the impact of Python and PyCUDA on scientist-developers.