Role of HPC in next-generation AI

Scale has been central to the success of deep learning with the availability of large-scale data and compute infrastructure. However, for further progress, scale has to be coupled with novel algorithms. Next-generation AI will be unsupervised, robust and adaptive. It will incorporate more structure and domain knowledge. Examples include tensors, graphs, physical laws, and simulations. I will describe efficient frameworks that enable developers to easily prototype such models, e.g., Tensorly to incorporate tensorized architectures, NVIDIA Isaac to incorporate physically valid simulations and NVIDIA RAPIDS for end-to-end data analytics. I will then lay out some outstanding problems in this area.