Multiway arrays, commonly referred to as higher-order tensors, are a natural data structure for representing multi-dimensional data and modeling processes consisting of composite interactions between factors. The tubal tensor framework [6, 1, 5] views a tensor as a ‘matrix of tubes’, where tubes are elements of a vector space supplemented with a binary, bilinear tubal multiplication, thus endowing the set of tubes with scalar-like properties that enable matrix mimetic tensor-tensor multiplication. From this perspective, tensors represent t-(tube-) linear mappings between Hilbert C*-modules over the algebra of tubes [3, 2], for example, a 3rd order tensor \(\smash {\tX \in \RR ^{m \xx p \xx n}}\) represents a t-linear mapping from \(\smash {\RR ^{p \xx 1 \xx n}}\) to \(\smash {\RR ^{m \xx 1 \xx n}}\), and the t-product of \(\smash {\tX }\) with a tensor \(\smash {\tY \in \RR ^{p \xx q \xx n}}\) is a tensor \(\smash {\tX * \tY \in \RR ^{m \xx q \xx n}}\) that represents the composition of the two mappings. The matrix mimetic nature of the t-product enables an almost direct translation of many matrix computations to the tensor setting in a way that preserve, to some extent, the theoretical properties of the original operations, e.g., perhaps most notable, the t-SVD which is a straightforward extension of the matrix SVD, and enjoys an Eckart-Young like optimality result for rank truncations of a tensor[2, 4, 5]. The extensive, still-growing set of matrix algorithms and tools, and the ease of their extension to tensors via the tubal framework, make it a powerful tool for dealing with multi-dimensional problems.

In many applications, tensor data is obtained by a finite set of observations of a multi-dimensional process evolving over a domain such as time or space. These processes are often modeled as elements within an infinite-dimensional Hilbert space.However, when tubes reside in an infinite-dimensional Hilbert space, the associated tubal algebra lacks certain properties present in the finite-dimensional case, such as a multiplicative identity and von Neumann regularity. This limitation hinders any direct extension of the tubal tensor framework to infinite-dimensional spaces, and, in particular, the tubal SVD is no longer viable.

In this work, we introduce the quasitubal tensor framework, an extension of the tubal tensor framework to tubal algebras defined on infinite-dimensional separable Hilbert spaces. Notably, we establish the existence of a quasitubal SVD and prove Eckart-Young optimality results for low-rank truncations of quasitubal SVD. With a strong theoretical basis, the quasitubal framework offers attractive approach for tackling multi-way problems in infinite-dimensional spaces.

Background. An order-\(N\) tensor \(\smash {\tX }\) over a field \(\FFF \) (either \(\CC \) or \(\RR \)) is an object in \(\smash {\FFF ^{d_1 \xx \cdots \xx d_N}}\). The line of research on tubal tensor algebra  [5, 1, 6, 4] views tensors ‘matrices of tubes’. For example, a 3rd order tensor \(\smash {\tX \in \FFF ^{m \xx p \xx n}}\) is considered as an \(m \xx p\) matrix over \(\smash {\FFF ^{n}}\) whose \(j,k\) (tubal) entry is \(\smash {\bs {x}_{jk} \in \FFF ^{n}}\). The t-product [6, 1, 5] of two tubes \(\smash {\bs {x},\bs {y} \in \FFF ^{n}}\) is defined as \(\smash {\bs {x} * \bs {y} = \mathtt {ifft}(\wh {\bs {x}} \odot \wh {\bs {y}})}\), where \(\smash {\wh {\bs {x}} = \mathtt {fft}(\bs {x})}\) is the Fourier transform of \(\bs {x}\), and \(\odot \) is the Hadamard product.

The mode-3 multiplication of \(\tX \) by a matrix \(\smash {\mat {A} \in \FFF ^{r \xx n}}\) is the tensor \(\smash {\tX \times _3 \mat {A} \in \FFF ^{m \xx p \xx r}}\) whose \(j,k\) tube fiber is given by \(\smash {\mat {A} \bs {x}_{jk} \in \FFF ^r}\). In particular, let \(\mat {F}\) be the \(n\xx n\) DFT matrix and define \(\smash {\wh {\tX } = \tX \xx _3 \mat {F}}\). The tensor-tensor t-product of \(\smash {\tX \in \FFF ^{m \xx p \xx n}, \tY \in \FFF ^{p \xx q \xx n}}\) is defined by \(\smash {\tX {*} \tY = (\wh {\tX } {\triangle } \wh {\tY })\xx _3 \mat {F}^{-1} }\) with \(\smash {\tZ = \tX \triangle \tY }\) a tensor such that \(\smash {\tZ _{:,:,j} = \tX _{:,:,j} \tY _{:,:,j} } \). Note that the t-product of two tubal-tensors is in-fact the multiplication of matrices over the tubal ring. More general version of the t-product is obtained by replacing \(\mat {F}\) with any invertible matrix \(\matM \) [2, 4] (so \(\smash {\wh {\tX } = \tX \xx _3 \matM }\)), resulting in the \(\mm \)-product, \(\smash {\tX \mm \tY = (\wh {\tX } \triangle \wh {\tY }) \xx _3 \matMi }\).

The \(p \xx p\) identity tensor \(\smash {\tI _p \in \FFF ^{p \xx p \xx n}} \) is such that \(\smash {\tX \mm \tI _p = \tX , \tI _p \mm \tY = \tY }\). The Hermitian adjoint of \(\smash {\tX \in \FFF ^{m \xx p \xx n}}\) is the tensor \(\smash {\tX ^H \in \FFF ^{p \xx m \xx n}}\) with \(\smash {\wh {\tX ^H}_{j,k,h} = \overline {\wh {x}_{k,j,h}}}\). A slice \(\smash {\slice {A} \in \FFF ^{p \xx 1 \xx n}}\) is \(\mm \) unit normalized if \(\smash {\slice {A}^H \mm \slice {A} = \bs {1}}\), and we say that \(\smash {\slice {A},\slice {B} \in \FFF ^{p \xx 1 \xx n}}\) are \(\mm \)-orthogonal if \(\smash {\slice {A}^H \mm \slice {B} = \bs {0}}\). A tensor \(\smash {\tU }\) is said to be \(\mm \)-unitary if \(\smash {\tU ^H * \tU = \tU \mm \tU ^H = \tI }\). The t-SVDM of \(\smash {\tX \in \FFF ^{m \xx p \xx n}}\) is a decomposition \(\smash {\tX = \tU \mm \tS \mm \tV ^H}\) where \(\smash {\tU \in \FFF ^{m \xx m \xx n}, \tV \in \FFF ^{p \xx p \xx n}}\) are \(\mm \)-unitary, and \(\smash {\tS \in \FFF ^{m \xx p \xx n}}\) is f-diagonal, i.e., \(\smash {\tS _{:,:,k}}\) are diagonal for all \(k\). The t-rank of \(\smash {\tX }\) under \(\mm \) [5, 2] is the number of non-zero diagonal tubes in \(\smash {\tS }\), and the multi-rank of \(\smash {\tX }\) under \(\mm \)[3, 4] is a vector \(\bs {\rho }\) of integers \(\rho _k = \rank (\thX _{:,:,k})\). Given \(r\leq \min (m,p)\), the t-rank \(r\) truncation of \(\smash {\tX }\) under \(\mm \) is the tensor \(\smash {\tX _r = \tU _{:,1:r,:} \mm \tS _{1:r,1:r,:} \mm {\tV _{:,1:r,:}}^H = \sum _{j=1}^r \slice {\tU }_j \mm \tS _{j,j,:} \mm {\slice {\tV }_{j}}^H }\) with \(\smash {\slice {\tU }_j = \tU _{:,j,:}}\) being the \(j\)th ‘column’ slice of \(\smash {\tU }\). For \(\bs {\rho } = (\rho _1,\ldots ,\rho _n)\) with \(\rho _k \leq \min (m,p)\), the multi-rank \(\bs {\rho }\) truncation of \(\smash {\tX }\) under \(\mm \) is the tensor \(\smash {\tX _{\bs {\rho }}}\) such that \(\smash {[\wh {\tX _{\bs {\rho }}}]_{:,:,k} = \wh {\tU }_{:,1:\rho _k,:} \triangle \wh {\tS }_{1:\rho _k,1:\rho _k,:} \triangle {\wh {\tV }_{:,1:\rho _k,:}}^H }\). The central result of the tubal framework is that the above truncations are optimal in the sense of Frobenius norm error, provided that \(\matM \) is a nonzero multiple of a unitary matrix. Formally, let \(\matM \) be a nonzero multiple of a unitary matrix and \(\smash {\tX \in \FFF ^{m \xx p \xx n}}\). If \(\smash {\tens {Y} \in \FFF ^{m \xx p \xx n}}\) is of t-rank \(r\) (respectively, multirank \(\bs {\rho }\)) under \(\mm \) then \(\smash {\FNorm {\tX - \tY } \geq \FNorm {\tX - \tX _r} }\)[5, 2] (respectively, \(\smash {\FNorm {\tX - \tY } \geq \FNorm {\tX - \tX _{\bs {\rho }}} }\)[4]).

In the above 3rd order example, each entry \(\smash {\bs {x}_{jk} \in \FFF ^{n}}\) of \(\smash {\tX }\) represents a function \(\smash {\bs {x}_{jk}\colon \Omega \to \FFF }\), where \(\Omega = [n] = \{1,\ldots ,n\}\) and \(\smash {x_{j,k,t} = \bs {x}_{jk}(t)}\). A common assumption in practice, is that the domain \(\Omega \) of \(\smash {\bs {x}_{jk}}\) is actually a compact subset of \(\RR \) and the values \(\smash {x_{j,k,h}}\) are point evaluations of \(\smash {\bs {x}_{jk}}\) on a grid \(\smash {t_1 \leq t_2 \leq \cdots \leq t_n \in \Omega }\) such that \(\smash {x_{j,k,h} = \bs {x}_{jk}(t_h)}\). Furthermore, it is possible to consider the functions \(\smash {\bs {x}_{jk}}\) as elements of a Hilbert space \(\smash {(\cH ,\dotph {\cdot ,\cdot })}\) in which vector addition and scalar multiplication are defined pointwise. In this case, we have a ‘matrix of functions’ in \(\smash {\cH }\) and we write \(\smash {\tX \in \cH ^{m \xx p}}\).

Matrices over Hilbert Spaces. Suppose that \(\smash {\cH }\) is a separable Hilbert space over \(\smash {\FFF }\), and let \(\smash {\{\bs {\phi }_j\}_{j \in \ZZ }}\) be an orthonormal basis in \(\smash {\cH }\). Then, the mapping \(\smash {\bs {x} \mapsto \Phi {\bs {x}} = \sum _j \dotph {\bs {x},\bs {\phi }_j} \bs {e}_j}\) where \(\smash {\bs {e}_j}\) is the \(j\)th standard basis vector in the space \(\smash {\ell _2}\) of square summable sequences with the usual dot product, is an isometry. Note that if \(\bs {a}, \bs {b} \in \ell _2\) then the elementwise multiplication \(\bs {a} \odot \bs {b} \) is also in \(\ell _2\). A natural extension of the \(\mm \) product to \(\smash {\cH }\) is given by \(\smash {\bs {x} \pp \bs {y} = \Phi ^{*}(\Phi {\bs {x}} \odot \Phi {\bs {y}})}\) where \(\smash {\Phi ^{*}}\) is the adjoint (and inverse) of \(\smash {\Phi }\). Let \(\smash {\tX \in \cH ^{m \xx p}}\) and define the mode-3 operation of \(\smash {\Phi }\) on \(\smash {\tX }\) as the tensor \(\smash {\thX = \tX \xx _3 \Phi \in \ell _2^{m \xx p}}\) with \(\smash {\wh {\bs {x}}_{jk} = \Phi {\bs {x}_{jk}}}\). Correspondingly, the the tensor-tensor \(\pp \)-product of \(\smash {\tX \in \cH ^{m \xx p}, \tY \in \cH ^{p \xx q}}\) is \(\smash {\tX \pp \tY = (\wh {\tX } \triangle \wh {\tY }) \xx _3 \Phi ^* }\).

The Challenge of Defining Tubal SVD in Infinite Dimensional Hilbert Space . Let \(\smash {\bs {x} \in \cH }\), then the operation \(T_{\bs {x}}\) defined by \(T_{\bs {x}} \bs {y} = \bs {x} \pp \bs {y}\) is a bounded linear operator on \(\smash {\cH }\). Furthermore, \(\smash {T_{\bs {x}}}\) is Hilbert-Schmidt operator since \(\sum _j \cHNormS {T_{\bs {x}} \bs {\phi }_j} = \sum _j \ltNormS {\wh {\bs {x}} \odot \bs {e}_j } = \sum _j |\wh {x}_j|^2 = \sum _j |\dotph {\bs {x},\bs {\phi }_j}|^2 = \cHNormS {\bs {x}} \). Thus, the a multiplicative identity in \(\smash {\cH }\) is impossible since it would imply that the identity operator is a Hilbert-Schmidt operator, in contradiction to the infinite-dimensionality of \(\smash {\cH }\). Direct consequences of this are that 1) there are no unit normalized slices in \(\smash {\cH ^p}\) 2) there are no \(\pp \)-unitary tensors in \(\smash {\cH ^{m \xx m}}\). Most importantly, no decomposition of the form \(\smash {\tX = \tU \pp \tS \pp \tV ^H}\) can be defined in \(\smash {\cH ^{m \xx p}}\) such that \(\smash {\tU \in \cH ^{m \xx m}, \tV \in \cH ^{p \xx p}}\) are isometries.

Quasitubal Framework. Consider the set \(\smash {\cH ^p \coloneqq \bigoplus _{j=1}^p \cH }\) of slices \(\smash {\slice {X} = (\bs {x}_1,\ldots ,\bs {x}_p)}\) with elementwise addition and \(\pp \)-product by \(\cH \) elements, e.g., \(\smash {\slice {X} \pp \bs {a} = \bs {a} \pp \slice {X} = (\bs {a} \pp \bs {x}_1,\ldots ,\bs {a} \pp \bs {x}_p)}\) and \(\smash {\slice {X} + \slice {Y} = (\bs {x}_1 + \bs {y}_1,\ldots ,\bs {x}_p + \bs {y}_p)}\) for \(\smash {\slice {X},\slice {Y} \in \cH ^p}\) and \(\smash {\bs {a} \in \cH }\). An operator \(T\colon \cH ^p \to \cH ^m\) is said to be t-linear (or \(\cH \)-linear) if \(T(\bs {a} \pp \slice {X}) = \bs {a} \pp T\slice {X}\) for all \(\smash {\bs {a} \in \cH , \slice {X} \in \cH ^p}\). Let \(\smash {L(\cH ^p,\cH ^m)}\) be the set of t-linear operators from \(\smash {\cH ^p}\) to \(\smash {\cH ^m}\). Note that such operators are necessarily bounded and linear over \(\FFF \), hence \(\smash {L(\cH ^p,\cH ^m) \subset B(\cH ^p,\cH ^m)}\).

Our theory is based on the following fundamental observations

We call \(\smash {L(\cH )}\) elements quasitubes due to their tubal representation in \(\smash {\ell _\infty }\). Respectively, operators in \(\smash {L(\cH ^p,\cH ^m)}\) are called quasitubal tensors as they retain a tubal tensor structure in the coordinates of the transform domain. We use the same notation for \(\smash {L(\cH )}\) and \(\smash {L(\cH ^p,\cH ^m)}\) operators as for elements in \(\smash {\cH ,\cH ^{m \xx p}}\), therefore, the \(\pp \) product of quasitubal tensors reads as composition of t-linear operators. While it is not possible to identify the space \(\smash {L(\cH ^p,\cH ^m)}\) with \(\smash {\cH ^{m \xx p}}\) (as in the finite-dimensional case), the notation is still compatible, valid and useful.

Given \(\smash {\slice {X} \in L(\cH )^p}\) we have \(\smash {|\slice {X}|_{L(\cH )^p}^2 = \dotmp {\slice {X},\slice {X}} }\) which is a non-negative element in a C*-algebra, hence has a unique square root \(\smash {|\slice {X}|_{L(\cH )^p}}\), and the real valued norm \(\smash {\GNorm {\slice {X}}_{L(\cH )^p} = \GNorm {|\slice {X}|_{L(\cH )^p}}}\). The induced “operator norm” of an \(\smash {m \xx p}\) quasitubal tensors is then \(\smash {\GNorm {\tX } = \sup _{|\slice {Y}|_{L(\cH )^p} = 1} \GNorm {\tX \pp \slice {Y}}_{L(\cH )^m}}\). Another consequence of the Hilbert C*-module structure over a unital C*-algebra, is the ability to define \(\pp \)-orthogonality and \(\pp \)-unitarity for quasitubal tensors similarly to the finite-dimensional case. With the above, the ground is set for construction of a quasitubal SVD:

The t-rank and multirank of a quasitensor \(\smash {\tX }\) under \(\pp \), as well as t-rank and multi-rank truncations, are defined similarly to the finite-dimensional case. And we have the main result:

Objects in \(\smash {\cH ^{m \xx p}}\) have the elementwise \(\cH \) norm: \(\smash {\cHNormS {\tX } = \sum _{j,k} \cHNormS {\bs {x}_{jk}}}\), which is an equivalent to the Frobenius norm in the finite-dimensional case. Consider \(\smash {\tX = \tU \pp \tS \pp \tV ^* \in \cH ^{m \xx p}}\), then \(\smash {\tS \in \cH ^{m \xx p}}\) and \(\smash {\cHNorm {\tX } = \cHNorm {\tS }}\). Importantly

Possible Applications. Due to the strong theoretical foundation of the quasitubal SVD, a promising line of research is the development of multivariate functional PCA, in a similar spirit to our previous work on the finite-dimensional settings [7]. Furthermore, the matrix mimetic nature of the platform, combined with the optimality results for low-rank truncations suggest that direct extensions of randomized algorithms for low-rank matrix approximations to the quasitubal setting are possible and should offer theoretical guarantees. This opens the door to computational speedups in modeling and simulations of multi-input multi-output dynamical systems where the quality of the approximation is about as crucial as the computational cost. We provide numerical examples for the application of the quasitubal framework to multivariate functional data analysis and signal processing, and demonstrate the potential of the framework for developing efficient tensor-based algorithms for such settings.

References

• [1]  K. Braman. Third-Order Tensors as Linear Operators on a Space of Matrices. Linear Algebra and its Applications, 433(7):1241–1253, 12 2010.

• [2]  E. Kernfeld, M. Kilmer, and S. Aeron. Tensor–tensor products with invertible linear transforms. Linear Algebra and its Applications, 485:545–570, Nov. 2015.

• [3]  M. E. Kilmer, K. Braman, N. Hao, and R. C. Hoover. Third-Order Tensors as Operators on Matrices: A Theoretical and Computational Framework with Applications in Imaging. SIAM Journal on Matrix Analysis and Applications, 34(1):148–172, 2 2013.

• [4]  M. E. Kilmer, L. Horesh, H. Avron, and E. Newman. Tensor-tensor algebra for optimal representation and compression of multiway data. Proceedings of the National Academy of Sciences, 118(28):e2015851118, July 2021.

• [5]  M. E. Kilmer and C. D. Martin. Factorization strategies for third-order tensors. Linear Algebra and its Applications, 435(3):641–658, 8 2011.

• [6]  M. E. Kilmer and C. D. M. Martin. Decomposing a tensor - are there analogues to the svd, lu, qr, and other matrix decompositions for tensors?, 11 2004.

• [7]  U. Mor, Y. Cohen, R. Valdés-Mas, D. Kviatcovsky, E. Elinav, and H. Avron. Dimensionality reduction of longitudinal ’omics data using modern tensor factorizations. PLOS Computational Biology, 18(7):e1010212, July 2022.