A Block-Sparse Tensor Train Format for Sample-Efficient High-Dimensional Polynomial Regression

Götte, Michael; Schneider, Reinhold; Trunschke, Philipp

Article

A Block-Sparse Tensor Train Format for Sample-Efficient High-Dimensional Polynomial Regression

Götte, Michael; Schneider, Reinhold; Trunschke, Philipp

FG Modellierung, Simulation und Optimierung in Natur- und Ingenieurwissenschaften

Low-rank tensors are an established framework for the parametrization of multivariate polynomials. We propose to extend this framework by including the concept of block-sparsity to efficiently parametrize homogeneous, multivariate polynomials with low-rank tensors. This provides a representation of general multivariate polynomials as a sum of homogeneous, multivariate polynomials, represented by block-sparse, low-rank tensors. We show that this sum can be concisely represented by a single block-sparse, low-rank tensor. We further prove cases, where low-rank tensors are particularly well suited by showing that for banded symmetric tensors of homogeneous polynomials the block sizes in the block-sparse multivariate polynomial space can be bounded independent of the number of variables. We showcase this format by applying it to high-dimensional least squares regression problems where it demonstrates improved computational resource utilization and sample efficiency.

sample efficiency homogeneous polynomials sparse tensor networks alternating least square empirical L2 approximation