High-Performance Tensor Contractions

Outline

Tensors (multi-dimensional arrays) appear in a wide range of scientific applications. While a range of software tools exist

for computations involving one- and two-dimensional arrays, i.e. vectors and matrices, the availability of high-performance software tools for tensors is much more limited. For instance, tensor contractions (TC) which can be though of a generalization of matrix-matrix multiplications to higher dimensions-- are a common and critical component of scientific computations, encountered in fields as diverse as machine learning [Abadi et 2015; Vasilache et al. 2014], spectral element methods [Tufo and Fischer 1999], quantum chemistry calculations [Harrison et al. 2016; Bartlett and Musial 2007], multidimensional Fourier transforms [Frigo and Johnson 2005; Pekurovsky 2012] and climate simulations [Drake et al. 1995]. Despite the close connection between matrix-matrix products (GEMM) and TCs, the performance of the latter is in general vastly inferior to that of an optimized GEMM.

The goal of this thesis is to close the performance gap between high-performance matrix implementations and those implementations currently available for higher-dimensional tensors.