Understanding the emergence of nuclear physics from the underlying fundamental theory of strong interactions with Quantum chromodynamics (QCD) requires the fastest supercomputers. We will describe the role of QCD in the evolution of our universe and discuss how we use the latest supercomputers, such as Summit at Oak Ridge National Laboratory and Sierra at Lawrence Livermore National Laboratory, to address basic questions such as why does the universe contain more matter than antimatter? Looking towards the exascale era, we can dream of tackling more complex questions related to the rate of protons fusing to helium in the sun and the state of matter in extreme conditions such as neutron stars. We'll explain why making the most of these new computers will require clever software to take advantage of the heterogeneous architectures. We'll also describe some advances in optimized use of GPUs, as well as management of the complex set of tasks required.